[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:23 dist-test-slave-bb3x krb5kdc[8176](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:23 dist-test-slave-bb3x krb5kdc[8176](info): set up 2 sockets
Sep 23 15:06:23 dist-test-slave-bb3x krb5kdc[8176](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 15:06:24.399837 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.062s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:24 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:24.528714 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:35941
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:35941
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:24.611328 8189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:24.611577 8189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:24.611617 8189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:24.612773 8189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:24.612840 8189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:24.612861 8189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:24.612882 8189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:24.612913 8189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:24.614404 8189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:35941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:35941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:24.614817 8189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:24.615069 8189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:24.620750 8197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:24.620792 8195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:24.620834 8194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:24.620896 8189 server_base.cc:1034] running on GCE node
I20260923 15:06:24.621376 8189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:24.621709 8189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:24.622911 8189 hybrid_clock.cc:648] HybridClock initialized: now 1790175984622881 us; error 46 us; skew 500 ppm
Sep 23 15:06:24 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:24.631283 8189 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:24.631877 8189 webserver.cc:460] Webserver started at http://127.7.250.126:41241/ using document root <none> and password file <none>
I20260923 15:06:24.632151 8189 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:24.632200 8189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:24.632305 8189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:24.633352 8189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "53b7639fb2d44abcbc19e348a85c04ef"
format_stamp: "Formatted at 2026-09-23 15:06:24 on dist-test-slave-bb3x"
server_key: "92691ceebd8966f2eeb6b224bfa03300"
server_key_iv: "3f50f637069393c90c9ef94692e4e27c"
server_key_version: "encryptionkey@0"
I20260923 15:06:24.633677 8189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "53b7639fb2d44abcbc19e348a85c04ef"
format_stamp: "Formatted at 2026-09-23 15:06:24 on dist-test-slave-bb3x"
server_key: "92691ceebd8966f2eeb6b224bfa03300"
server_key_iv: "3f50f637069393c90c9ef94692e4e27c"
server_key_version: "encryptionkey@0"
I20260923 15:06:24.635221 8189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:24.636060 8204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:24.636324 8189 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:24.636402 8189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "53b7639fb2d44abcbc19e348a85c04ef"
format_stamp: "Formatted at 2026-09-23 15:06:24 on dist-test-slave-bb3x"
server_key: "92691ceebd8966f2eeb6b224bfa03300"
server_key_iv: "3f50f637069393c90c9ef94692e4e27c"
server_key_version: "encryptionkey@0"
I20260923 15:06:24.636497 8189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:24.653865 8189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:24.656204 8189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:24.656327 8189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:24.660588 8189 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:35941
I20260923 15:06:24.660796 8256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:35941 every 8 connection(s)
I20260923 15:06:24.661062 8189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:24.662196 8257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:24.664018 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8189
I20260923 15:06:24.664136 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:24.664917 8169 external_mini_cluster.cc:1428] Setting key b84336c497a34cd8c49c980e958a192a
I20260923 15:06:24.665329 8257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef: Bootstrap starting.
I20260923 15:06:24.666425 8257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:24.666810 8257 log.cc:824] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:24.667621 8257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef: No bootstrap required, opened a new log
I20260923 15:06:24.669935 8257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b7639fb2d44abcbc19e348a85c04ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 35941 } }
I20260923 15:06:24.670069 8257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:24.670092 8257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b7639fb2d44abcbc19e348a85c04ef, State: Initialized, Role: FOLLOWER
I20260923 15:06:24.670248 8257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b7639fb2d44abcbc19e348a85c04ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 35941 } }
I20260923 15:06:24.670378 8257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:24.670425 8257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:24.670475 8257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:24.671420 8257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b7639fb2d44abcbc19e348a85c04ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 35941 } }
I20260923 15:06:24.671543 8257 leader_election.cc:302] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b7639fb2d44abcbc19e348a85c04ef; no voters:
I20260923 15:06:24.671768 8257 leader_election.cc:288] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:24.671784 8262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:24.671918 8262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [term 1 LEADER]: Becoming Leader. State: Replica: 53b7639fb2d44abcbc19e348a85c04ef, State: Running, Role: LEADER
I20260923 15:06:24.672021 8262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b7639fb2d44abcbc19e348a85c04ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 35941 } }
I20260923 15:06:24.672145 8257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:24.672425 8262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b7639fb2d44abcbc19e348a85c04ef. Latest consensus state: current_term: 1 leader_uuid: "53b7639fb2d44abcbc19e348a85c04ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b7639fb2d44abcbc19e348a85c04ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 35941 } } }
I20260923 15:06:24.672511 8262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:24.672426 8263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b7639fb2d44abcbc19e348a85c04ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b7639fb2d44abcbc19e348a85c04ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 35941 } } }
I20260923 15:06:24.672578 8263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:24.673390 8267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:24.673959 8267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:24.675670 8267 catalog_manager.cc:1347] Generated new cluster ID: 9cdf999ac62b45fa89b4ec224d97f3a1
I20260923 15:06:24.675743 8267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:24.680224 8267 catalog_manager.cc:1370] Generated new certificate authority record
Sep 23 15:06:24 dist-test-slave-bb3x krb5kdc[8176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:24.681013 8267 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:24.686528 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:24.666370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40104 (local address 127.7.250.126:35941)
0923 15:06:24.666687 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:24.666698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:24.666982 (+ 284us) server_negotiation.cc:407] Connection header received
0923 15:06:24.669930 (+ 2948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:24.669939 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:24.669996 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:24.670223 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:24.672298 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:24.674138 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:24.675838 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:24.676488 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:24.676598 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:24.681290 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:24.681294 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:24.681295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:24.681308 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:24.684250 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:24.684818 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:24.684822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:24.684823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:24.684931 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:24.685239 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:24.685240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:24.685241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:24.685545 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:24.685727 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:24.686031 (+ 304us) server_negotiation.cc:299] Negotiation successful
0923 15:06:24.686292 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":112,"threads_started":1}
I20260923 15:06:24.687140 8267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b7639fb2d44abcbc19e348a85c04ef: Generated new TSK 0
I20260923 15:06:24.687340 8267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:24.720995 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:35941
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:24.803790 8283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:24.804032 8283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:24.804075 8283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:24.805260 8283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:24.805333 8283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:24.805380 8283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:24.807060 8283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:35941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:24.807364 8283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:24.807608 8283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:24.814059 8288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:24.814491 8283 server_base.cc:1034] running on GCE node
W20260923 15:06:24.814055 8289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:24.814294 8291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:24.814762 8283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:24.815018 8283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:24.816201 8283 hybrid_clock.cc:648] HybridClock initialized: now 1790175984816179 us; error 35 us; skew 500 ppm
Sep 23 15:06:24 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:24.819069 8283 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:24.819577 8283 webserver.cc:460] Webserver started at http://127.7.250.65:37107/ using document root <none> and password file <none>
I20260923 15:06:24.819828 8283 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:24.819898 8283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:24.820037 8283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:24.821084 8283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "407fa53582454e33b0af13c4e2164c7c"
format_stamp: "Formatted at 2026-09-23 15:06:24 on dist-test-slave-bb3x"
server_key: "ca420955fff50e72e4317ab2e88d3ad3"
server_key_iv: "381625694c9a2407fad652787667e3e4"
server_key_version: "encryptionkey@0"
I20260923 15:06:24.821452 8283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "407fa53582454e33b0af13c4e2164c7c"
format_stamp: "Formatted at 2026-09-23 15:06:24 on dist-test-slave-bb3x"
server_key: "ca420955fff50e72e4317ab2e88d3ad3"
server_key_iv: "381625694c9a2407fad652787667e3e4"
server_key_version: "encryptionkey@0"
I20260923 15:06:24.822906 8283 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:24.823668 8298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:24.823915 8283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:24.824019 8283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "407fa53582454e33b0af13c4e2164c7c"
format_stamp: "Formatted at 2026-09-23 15:06:24 on dist-test-slave-bb3x"
server_key: "ca420955fff50e72e4317ab2e88d3ad3"
server_key_iv: "381625694c9a2407fad652787667e3e4"
server_key_version: "encryptionkey@0"
I20260923 15:06:24.824092 8283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:24.864428 8283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:24.867167 8283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:24.867295 8283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:24.867498 8283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:24.867870 8283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:24.867977 8283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:24.868005 8283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:24.868018 8283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:24.874933 8283 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:37441
I20260923 15:06:24.875015 8411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:37441 every 8 connection(s)
I20260923 15:06:24.875399 8283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:24.876722 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8283
I20260923 15:06:24.876899 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:24.877081 8169 external_mini_cluster.cc:1428] Setting key e068237fd5df2458ce1b5098c2a710f9
Sep 23 15:06:24 dist-test-slave-bb3x krb5kdc[8176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175984, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
I20260923 15:06:24.887173 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:24.876310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:48431 (local address 127.7.250.126:35941)
0923 15:06:24.876400 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:24.876402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:24.877273 (+ 871us) server_negotiation.cc:407] Connection header received
0923 15:06:24.877311 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:24.877312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:24.877326 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:24.877404 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:24.878585 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:24.879476 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:24.880952 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:24.881381 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:24.881465 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:24.883662 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:24.883664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:24.883665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:24.883674 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:24.885415 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:24.885830 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:24.885832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:24.885832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:24.885900 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:24.886215 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:24.886217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:24.886217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:24.886418 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:24.886502 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:24.886916 (+ 414us) server_negotiation.cc:299] Negotiation successful
0923 15:06:24.887031 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:06:24.887245 8414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:24.876276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:35941 (local address 127.7.250.65:48431)
0923 15:06:24.876457 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:24.876505 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:24.877109 (+ 604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:24.877553 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:24.877558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:24.878221 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:24.878479 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:24.878484 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:24.879588 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:24.879589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:24.880843 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:24.880846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:24.881468 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:24.881469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:24.881574 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:24.882126 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:24.882134 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:24.883562 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:24.885515 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:24.885522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:24.885523 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:24.885742 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:24.886004 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:24.886005 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:24.886005 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:24.886118 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:24.886550 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:24.886559 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:24.886709 (+ 150us) client_negotiation.cc:769] Sending connection context
0923 15:06:24.886888 (+ 179us) client_negotiation.cc:240] Negotiation successful
0923 15:06:24.887052 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260923 15:06:24.888127 8412 heartbeater.cc:347] Connected to a master server at 127.7.250.126:35941
I20260923 15:06:24.888255 8412 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:24.888489 8412 heartbeater.cc:511] Master 127.7.250.126:35941 requested a full tablet report, sending...
I20260923 15:06:24.889223 8221 ts_manager.cc:194] Registered new tserver with Master: 407fa53582454e33b0af13c4e2164c7c (127.7.250.65:37441)
I20260923 15:06:24.889943 8221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:48431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:24.909909 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:35941
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:24.995448 8417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:24.995719 8417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:24.995765 8417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:24.996981 8417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:24.997212 8417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:24.997329 8417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:24.999001 8417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:35941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:24.999302 8417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:24.999548 8417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:25.004887 8417 server_base.cc:1034] running on GCE node
W20260923 15:06:25.004887 8422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:25.004925 8423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:25.005034 8425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:25.005306 8417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:25.005574 8417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:25.006767 8417 hybrid_clock.cc:648] HybridClock initialized: now 1790175985006734 us; error 50 us; skew 500 ppm
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:25.009636 8417 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:25.010169 8417 webserver.cc:460] Webserver started at http://127.7.250.66:32831/ using document root <none> and password file <none>
I20260923 15:06:25.010439 8417 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:25.010490 8417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:25.010589 8417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:25.011597 8417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "508222e37af84957b91aa4810ff0bbb1"
format_stamp: "Formatted at 2026-09-23 15:06:25 on dist-test-slave-bb3x"
server_key: "f42fa50b5616be6a27758403771bc02e"
server_key_iv: "ffa9091671c409b7b07d966eaf360ad1"
server_key_version: "encryptionkey@0"
I20260923 15:06:25.011904 8417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "508222e37af84957b91aa4810ff0bbb1"
format_stamp: "Formatted at 2026-09-23 15:06:25 on dist-test-slave-bb3x"
server_key: "f42fa50b5616be6a27758403771bc02e"
server_key_iv: "ffa9091671c409b7b07d966eaf360ad1"
server_key_version: "encryptionkey@0"
I20260923 15:06:25.013360 8417 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:25.014251 8432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:25.014510 8417 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:25.014597 8417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "508222e37af84957b91aa4810ff0bbb1"
format_stamp: "Formatted at 2026-09-23 15:06:25 on dist-test-slave-bb3x"
server_key: "f42fa50b5616be6a27758403771bc02e"
server_key_iv: "ffa9091671c409b7b07d966eaf360ad1"
server_key_version: "encryptionkey@0"
I20260923 15:06:25.014657 8417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:25.028400 8417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:25.030850 8417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:25.030972 8417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:25.031157 8417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:25.031427 8417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:25.031459 8417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:25.031482 8417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:25.031494 8417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:25.037251 8417 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:38613
I20260923 15:06:25.037333 8545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:38613 every 8 connection(s)
I20260923 15:06:25.037652 8417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:25.045439 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8417
I20260923 15:06:25.045545 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:25.045831 8169 external_mini_cluster.cc:1428] Setting key de058f217c3c94400d5fae295d31ea04
I20260923 15:06:25.048130 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:25.038471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:46035 (local address 127.7.250.126:35941)
0923 15:06:25.038537 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:25.038538 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:25.038757 (+ 219us) server_negotiation.cc:407] Connection header received
0923 15:06:25.039335 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:25.039337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:25.039351 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:25.039430 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:25.040444 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.040949 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.041952 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.042149 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.042234 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:25.044624 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:25.044626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:25.044627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:25.044636 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:25.046454 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.046955 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.046957 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.046957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.047027 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.047331 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.047332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.047332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.047515 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:25.047590 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:25.047868 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 15:06:25.047975 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:06:25.048180 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:25.038447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:35941 (local address 127.7.250.66:46035)
0923 15:06:25.038622 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:25.038655 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:25.039218 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:25.039531 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:25.039535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:25.040120 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:25.040341 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:25.040346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.041042 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.041043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:25.041846 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:25.041848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.042244 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.042245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:25.042399 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:25.042988 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:25.042997 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:25.044502 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:25.046565 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:25.046574 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:25.046576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:25.046871 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:25.047133 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:25.047134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:25.047134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:25.047257 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:25.047583 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:25.047588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:25.047719 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 15:06:25.047821 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 15:06:25.047958 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260923 15:06:25.049036 8546 heartbeater.cc:347] Connected to a master server at 127.7.250.126:35941
I20260923 15:06:25.049160 8546 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:25.049383 8546 heartbeater.cc:511] Master 127.7.250.126:35941 requested a full tablet report, sending...
I20260923 15:06:25.049824 8221 ts_manager.cc:194] Registered new tserver with Master: 508222e37af84957b91aa4810ff0bbb1 (127.7.250.66:38613)
I20260923 15:06:25.050449 8221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:46035
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:25.074431 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:35941
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:25.164697 8551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:25.164937 8551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:25.164976 8551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:25.166348 8551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:25.166424 8551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:25.166473 8551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:25.168220 8551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:35941
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:25.168502 8551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:25.168744 8551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:25.174024 8559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:25.174024 8556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:25.174150 8551 server_base.cc:1034] running on GCE node
W20260923 15:06:25.174078 8557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:25.174515 8551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:25.174721 8551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:25.175872 8551 hybrid_clock.cc:648] HybridClock initialized: now 1790175985175853 us; error 41 us; skew 500 ppm
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:25.178479 8551 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:25.178941 8551 webserver.cc:460] Webserver started at http://127.7.250.67:45245/ using document root <none> and password file <none>
I20260923 15:06:25.179203 8551 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:25.179280 8551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:25.179428 8551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:25.180491 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "798c4433123142989f8294a0a6efb05f"
format_stamp: "Formatted at 2026-09-23 15:06:25 on dist-test-slave-bb3x"
server_key: "a8b120068355d6b3cf56f22614238da0"
server_key_iv: "305cd7f5ebe68a1553a27046bde62f0e"
server_key_version: "encryptionkey@0"
I20260923 15:06:25.180855 8551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "798c4433123142989f8294a0a6efb05f"
format_stamp: "Formatted at 2026-09-23 15:06:25 on dist-test-slave-bb3x"
server_key: "a8b120068355d6b3cf56f22614238da0"
server_key_iv: "305cd7f5ebe68a1553a27046bde62f0e"
server_key_version: "encryptionkey@0"
I20260923 15:06:25.182327 8551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 15:06:25.183115 8566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:25.183302 8551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:25.183393 8551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "798c4433123142989f8294a0a6efb05f"
format_stamp: "Formatted at 2026-09-23 15:06:25 on dist-test-slave-bb3x"
server_key: "a8b120068355d6b3cf56f22614238da0"
server_key_iv: "305cd7f5ebe68a1553a27046bde62f0e"
server_key_version: "encryptionkey@0"
I20260923 15:06:25.183477 8551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:25.232228 8551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:25.234745 8551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:25.234931 8551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:25.235189 8551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:25.235510 8551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:25.235567 8551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:25.235608 8551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:25.235637 8551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:25.242105 8551 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:36745
I20260923 15:06:25.242197 8679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:36745 every 8 connection(s)
I20260923 15:06:25.242537 8551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:25.250221 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8551
I20260923 15:06:25.250334 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:25.250506 8169 external_mini_cluster.cc:1428] Setting key 829b0a2ca97ffc99e57cd80c3e09a78a
I20260923 15:06:25.252741 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:25.243278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:44345 (local address 127.7.250.126:35941)
0923 15:06:25.243370 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:25.243373 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:25.243566 (+ 193us) server_negotiation.cc:407] Connection header received
0923 15:06:25.244128 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:25.244129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:25.244143 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:25.244200 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:25.245230 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.245738 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.246779 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.246997 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.247083 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:25.249416 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:25.249418 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:25.249418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:25.249435 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:25.251178 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.251666 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.251667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.251668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.251731 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.252036 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.252037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.252037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.252161 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:25.252218 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:25.252545 (+ 327us) server_negotiation.cc:299] Negotiation successful
0923 15:06:25.252612 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:06:25.253021 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:25.243243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:35941 (local address 127.7.250.67:44345)
0923 15:06:25.243430 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:25.243475 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:25.244009 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:25.244277 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:25.244281 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:25.244921 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:25.245120 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:25.245125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.245845 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.245847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:25.246681 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:25.246683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.247096 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.247097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:25.247255 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:25.247839 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:25.247850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:25.249298 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:25.251254 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:25.251276 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:25.251278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:25.251588 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:25.251865 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:25.251866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:25.251866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:25.251970 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:25.252252 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:25.252259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:25.252408 (+ 149us) client_negotiation.cc:769] Sending connection context
0923 15:06:25.252519 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:06:25.252724 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260923 15:06:25.253575 8680 heartbeater.cc:347] Connected to a master server at 127.7.250.126:35941
I20260923 15:06:25.253727 8680 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:25.253966 8680 heartbeater.cc:511] Master 127.7.250.126:35941 requested a full tablet report, sending...
I20260923 15:06:25.254487 8221 ts_manager.cc:194] Registered new tserver with Master: 798c4433123142989f8294a0a6efb05f (127.7.250.67:36745)
I20260923 15:06:25.254966 8221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:44345
I20260923 15:06:25.262917 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:25.288338 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:25.278930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40764 (local address 127.7.250.65:37441)
0923 15:06:25.279075 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:25.279081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:25.279096 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:06:25.279207 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:25.279209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:25.279222 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:25.279299 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:25.279954 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.280774 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.281681 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.282058 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.282128 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:25.285113 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:25.285125 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:25.285125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:25.285132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:25.286950 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.287382 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.287386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.287386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.287465 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.287652 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.287653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.287653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.287863 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:25.287988 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:25.288078 (+ 90us) server_negotiation.cc:299] Negotiation successful
0923 15:06:25.288155 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
W20260923 15:06:25.288902 8326 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40764
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8176](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:25.312700 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:25.305124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40772 (local address 127.7.250.65:37441)
0923 15:06:25.305221 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:25.305223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:25.305235 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:25.305316 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:25.305317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:25.305330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:25.305384 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:25.306012 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.306889 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.307626 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:25.307861 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:25.307934 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:25.310217 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:25.310218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:25.310219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:25.310300 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:25.311767 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.312074 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.312075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.312075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.312125 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:25.312286 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:25.312287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:25.312287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:25.312380 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:25.312471 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:25.312542 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 15:06:25.312604 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:06:25.313496 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8283
I20260923 15:06:25.318436 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8417
I20260923 15:06:25.323522 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8551
I20260923 15:06:25.327843 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8189
2026-09-23T15:06:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2027 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8694](info): set up 2 sockets
Sep 23 15:06:25 dist-test-slave-bb3x krb5kdc[8694](info): commencing operation
krb5kdc: starting...
W20260923 15:06:26.371402 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:26 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:26.491559 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39195
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:26.573459 8707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:26.573734 8707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:26.573771 8707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:26.575012 8707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:26.575080 8707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:26.575105 8707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:26.575127 8707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:26.575156 8707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:26.576642 8707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:26.576931 8707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:26.577163 8707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:26.582526 8712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:26.582540 8713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:26.582557 8707 server_base.cc:1034] running on GCE node
W20260923 15:06:26.582605 8715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:26.582984 8707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:26.583258 8707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:26.584424 8707 hybrid_clock.cc:648] HybridClock initialized: now 1790175986584398 us; error 40 us; skew 500 ppm
Sep 23 15:06:26 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:26.587218 8707 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:26.587667 8707 webserver.cc:460] Webserver started at http://127.7.250.126:36007/ using document root <none> and password file <none>
I20260923 15:06:26.587939 8707 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:26.587990 8707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:26.588173 8707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:26.589337 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "470f41c13c554d08b4d15ce6ab8e003d"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "eb9332733aa133a3952adf73d5984329"
server_key_iv: "83d3f7bec1f7388754c58a295f679867"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.589733 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "470f41c13c554d08b4d15ce6ab8e003d"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "eb9332733aa133a3952adf73d5984329"
server_key_iv: "83d3f7bec1f7388754c58a295f679867"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.591284 8707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:26.592223 8722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:26.592520 8707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:26.592773 8707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "470f41c13c554d08b4d15ce6ab8e003d"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "eb9332733aa133a3952adf73d5984329"
server_key_iv: "83d3f7bec1f7388754c58a295f679867"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.592868 8707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:26.624831 8707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:26.627290 8707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:26.627485 8707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:26.632052 8707 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:39195
I20260923 15:06:26.632094 8774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:39195 every 8 connection(s)
I20260923 15:06:26.632529 8707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:26.633436 8775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:26.635697 8775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d: Bootstrap starting.
I20260923 15:06:26.636235 8775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:26.636469 8775 log.cc:824] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:26.636989 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8707
I20260923 15:06:26.637082 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:26.637257 8169 external_mini_cluster.cc:1428] Setting key c1b91859108b1989bf00f559ffb26903
I20260923 15:06:26.637503 8775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d: No bootstrap required, opened a new log
I20260923 15:06:26.640173 8775 raft_consensus.cc:348] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470f41c13c554d08b4d15ce6ab8e003d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39195 } }
I20260923 15:06:26.640401 8775 raft_consensus.cc:374] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:26.640452 8775 raft_consensus.cc:729] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 470f41c13c554d08b4d15ce6ab8e003d, State: Initialized, Role: FOLLOWER
I20260923 15:06:26.640630 8775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470f41c13c554d08b4d15ce6ab8e003d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39195 } }
I20260923 15:06:26.640734 8775 raft_consensus.cc:388] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:26.640769 8775 raft_consensus.cc:482] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:26.640823 8775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:26.641626 8775 raft_consensus.cc:504] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470f41c13c554d08b4d15ce6ab8e003d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39195 } }
I20260923 15:06:26.641757 8775 leader_election.cc:302] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 470f41c13c554d08b4d15ce6ab8e003d; no voters:
I20260923 15:06:26.641917 8775 leader_election.cc:288] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:26.641966 8780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:26.642145 8780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [term 1 LEADER]: Becoming Leader. State: Replica: 470f41c13c554d08b4d15ce6ab8e003d, State: Running, Role: LEADER
I20260923 15:06:26.642177 8775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:26.642249 8780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470f41c13c554d08b4d15ce6ab8e003d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39195 } }
I20260923 15:06:26.642585 8780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 470f41c13c554d08b4d15ce6ab8e003d. Latest consensus state: current_term: 1 leader_uuid: "470f41c13c554d08b4d15ce6ab8e003d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470f41c13c554d08b4d15ce6ab8e003d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39195 } } }
I20260923 15:06:26.642679 8780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:26.642594 8782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "470f41c13c554d08b4d15ce6ab8e003d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "470f41c13c554d08b4d15ce6ab8e003d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39195 } } }
I20260923 15:06:26.642824 8782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:26.642975 8786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:26.643529 8786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:26.645664 8786 catalog_manager.cc:1347] Generated new cluster ID: d29557a52fe34609b26194aec5874e28
I20260923 15:06:26.645725 8786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:26 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:26.651276 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:26.637997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57494 (local address 127.7.250.126:39195)
0923 15:06:26.638160 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:26.638173 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:26.638224 (+ 51us) server_negotiation.cc:407] Connection header received
0923 15:06:26.639288 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:26.639299 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:26.639371 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:26.639550 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:26.640272 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:26.642172 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:26.643554 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:26.644442 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:26.644608 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:26.647591 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:26.647595 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:26.647595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:26.647612 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:26.649538 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:26.649944 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:26.649950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:26.649950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:26.650041 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:26.650361 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:26.650362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:26.650363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:26.650623 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:26.650812 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:26.650891 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 15:06:26.651061 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260923 15:06:26.676600 8786 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:26.677191 8786 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:26.686513 8786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 470f41c13c554d08b4d15ce6ab8e003d: Generated new TSK 0
I20260923 15:06:26.686756 8786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:26.720980 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39195
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:26.806087 8801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:26.806326 8801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:26.806352 8801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:26.807511 8801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:26.807551 8801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:26.807578 8801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:26.809036 8801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39195
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8801
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:26.809237 8801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:26.809453 8801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:26.814457 8809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:26.814467 8806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:26.814467 8807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:26.814716 8801 server_base.cc:1034] running on GCE node
I20260923 15:06:26.814949 8801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:26.815191 8801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:26.816361 8801 hybrid_clock.cc:648] HybridClock initialized: now 1790175986816354 us; error 35 us; skew 500 ppm
Sep 23 15:06:26 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:26.819012 8801 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:26.819446 8801 webserver.cc:460] Webserver started at http://127.7.250.65:37453/ using document root <none> and password file <none>
I20260923 15:06:26.819674 8801 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:26.819741 8801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:26.819890 8801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:26.820930 8801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "989917866f2946f98dd661d737d0d670"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "ad428b275a7e7e85922d0897dd6fd45c"
server_key_iv: "de6e4c971ee81d4f5e8b2ea7d17e1083"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.821295 8801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "989917866f2946f98dd661d737d0d670"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "ad428b275a7e7e85922d0897dd6fd45c"
server_key_iv: "de6e4c971ee81d4f5e8b2ea7d17e1083"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.822670 8801 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:26.823663 8816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:26.823916 8801 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:26.824045 8801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "989917866f2946f98dd661d737d0d670"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "ad428b275a7e7e85922d0897dd6fd45c"
server_key_iv: "de6e4c971ee81d4f5e8b2ea7d17e1083"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.824148 8801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:26.835999 8801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:26.837957 8801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:26.838119 8801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:26.838405 8801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:26.838717 8801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:26.838771 8801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:26.838814 8801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:26.838840 8801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:26.845811 8801 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:34791
I20260923 15:06:26.845883 8929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:34791 every 8 connection(s)
I20260923 15:06:26.846216 8801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:06:26 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:26.855409 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:26.846967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:39983 (local address 127.7.250.126:39195)
0923 15:06:26.847036 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:26.847038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:26.847237 (+ 199us) server_negotiation.cc:407] Connection header received
0923 15:06:26.848148 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:26.848149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:26.848158 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:26.848224 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:26.848795 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:26.849299 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:26.850408 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:26.850603 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:26.850658 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:26.852730 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:26.852731 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:26.852731 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:26.852736 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:26.853947 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:26.854361 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:26.854361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:26.854362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:26.854411 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:26.854792 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:26.854793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:26.854793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:26.854902 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:26.854953 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:26.855250 (+ 297us) server_negotiation.cc:299] Negotiation successful
0923 15:06:26.855344 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:06:26.855700 8932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:26.846963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39195 (local address 127.7.250.65:39983)
0923 15:06:26.847109 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:26.847150 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:26.848063 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:26.848283 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:26.848286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:26.848534 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:26.848723 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:26.848727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:26.849378 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:26.849379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:26.850319 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:26.850322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:26.850673 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:26.850674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:26.850792 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:26.851322 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:26.851337 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:26.852653 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:26.854018 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:26.854024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:26.854026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:26.854231 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:26.854491 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:26.854492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:26.854492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:26.854719 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:26.854963 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:26.854968 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:26.855104 (+ 136us) client_negotiation.cc:769] Sending connection context
0923 15:06:26.855212 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 15:06:26.855349 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260923 15:06:26.856353 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8801
I20260923 15:06:26.856371 8930 heartbeater.cc:347] Connected to a master server at 127.7.250.126:39195
I20260923 15:06:26.856432 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:26.856487 8930 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:26.856618 8169 external_mini_cluster.cc:1428] Setting key 8768a10d705454afb80722bdf745fe76
I20260923 15:06:26.856794 8930 heartbeater.cc:511] Master 127.7.250.126:39195 requested a full tablet report, sending...
I20260923 15:06:26.857373 8738 ts_manager.cc:194] Registered new tserver with Master: 989917866f2946f98dd661d737d0d670 (127.7.250.65:34791)
I20260923 15:06:26.858047 8738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:39983
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:26.881785 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39195
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:26.968011 8935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:26.968217 8935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:26.968241 8935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:26.969318 8935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:26.969352 8935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:26.969378 8935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:26.970824 8935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39195
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:26.971040 8935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:26.971220 8935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:26.976219 8941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:26.976322 8940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:26.976307 8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:26.976559 8935 server_base.cc:1034] running on GCE node
I20260923 15:06:26.976841 8935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:26.977044 8935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:26.978240 8935 hybrid_clock.cc:648] HybridClock initialized: now 1790175986978196 us; error 59 us; skew 500 ppm
Sep 23 15:06:26 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:26.981171 8935 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:26.981700 8935 webserver.cc:460] Webserver started at http://127.7.250.66:32859/ using document root <none> and password file <none>
I20260923 15:06:26.981943 8935 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:26.982023 8935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:26.982172 8935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:26.983223 8935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "ccdd60b35952419593b08bf1e477d594"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "94abcb3b073cd880c266279812c1873e"
server_key_iv: "46877becbfb053cc30f764ce2e271075"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.983584 8935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "ccdd60b35952419593b08bf1e477d594"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "94abcb3b073cd880c266279812c1873e"
server_key_iv: "46877becbfb053cc30f764ce2e271075"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.984900 8935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:06:26.985601 8950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:26.985769 8935 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:26.985872 8935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "ccdd60b35952419593b08bf1e477d594"
format_stamp: "Formatted at 2026-09-23 15:06:26 on dist-test-slave-bb3x"
server_key: "94abcb3b073cd880c266279812c1873e"
server_key_iv: "46877becbfb053cc30f764ce2e271075"
server_key_version: "encryptionkey@0"
I20260923 15:06:26.985944 8935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:26.992949 8935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:26.995050 8935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:26.995217 8935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:26.995462 8935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:26.995832 8935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:26.995885 8935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:26.995929 8935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:26.995956 8935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:27.001982 8935 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:35837
I20260923 15:06:27.002033 9063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:35837 every 8 connection(s)
I20260923 15:06:27.002395 8935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:06:27.007004 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 8935
I20260923 15:06:27.007107 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:27.007282 8169 external_mini_cluster.cc:1428] Setting key be81e1112d16f2aae84c0db238ebad14
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:27.012841 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.003094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:38163 (local address 127.7.250.126:39195)
0923 15:06:27.003190 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.003192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.003537 (+ 345us) server_negotiation.cc:407] Connection header received
0923 15:06:27.004365 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.004367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.004377 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.004440 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:27.005085 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.005593 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.006894 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.007279 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.007371 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.009571 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:27.009573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:27.009573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:27.009583 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:27.011310 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.011788 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.011790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.011790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.011854 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.012130 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.012131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.012132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.012260 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:27.012341 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.012648 (+ 307us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.012734 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 15:06:27.013067 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.003068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39195 (local address 127.7.250.66:38163)
0923 15:06:27.003267 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.003308 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.004257 (+ 949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.004508 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.004511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.004812 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:27.005002 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.005006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.005748 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.005749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.006747 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.006749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.007406 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.007408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.007555 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.008070 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:27.008087 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:27.009458 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:27.011420 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:27.011428 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:27.011430 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:27.011686 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:27.011936 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:27.011937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:27.011938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:27.012054 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:27.012346 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:27.012353 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:27.012507 (+ 154us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.012624 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.012807 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260923 15:06:27.013746 9064 heartbeater.cc:347] Connected to a master server at 127.7.250.126:39195
I20260923 15:06:27.013833 9064 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:27.014036 9064 heartbeater.cc:511] Master 127.7.250.126:39195 requested a full tablet report, sending...
I20260923 15:06:27.014462 8738 ts_manager.cc:194] Registered new tserver with Master: ccdd60b35952419593b08bf1e477d594 (127.7.250.66:35837)
I20260923 15:06:27.014981 8738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:38163
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:27.034127 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39195
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:27.118626 9069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:27.118839 9069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:27.118865 9069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:27.120007 9069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:27.120044 9069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:27.120071 9069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:27.121568 9069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39195
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:27.121780 9069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:27.121961 9069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:27.127583 9075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:27.127583 9074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:27.127602 9069 server_base.cc:1034] running on GCE node
W20260923 15:06:27.127923 9077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:27.128139 9069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:27.128404 9069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:27.129582 9069 hybrid_clock.cc:648] HybridClock initialized: now 1790175987129569 us; error 33 us; skew 500 ppm
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175987, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:27.132668 9069 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:27.133178 9069 webserver.cc:460] Webserver started at http://127.7.250.67:42149/ using document root <none> and password file <none>
I20260923 15:06:27.133446 9069 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:27.133515 9069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:27.133662 9069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:27.134800 9069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "90711cf27a774e7b887742d823d51f26"
format_stamp: "Formatted at 2026-09-23 15:06:27 on dist-test-slave-bb3x"
server_key: "fb8ca0f62b4cf5a86e557f927a67ad34"
server_key_iv: "426f7d68aeaf6a06cf0f6c5458602796"
server_key_version: "encryptionkey@0"
I20260923 15:06:27.135162 9069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "90711cf27a774e7b887742d823d51f26"
format_stamp: "Formatted at 2026-09-23 15:06:27 on dist-test-slave-bb3x"
server_key: "fb8ca0f62b4cf5a86e557f927a67ad34"
server_key_iv: "426f7d68aeaf6a06cf0f6c5458602796"
server_key_version: "encryptionkey@0"
I20260923 15:06:27.136582 9069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:27.137734 9084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:27.137995 9069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:27.138142 9069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "90711cf27a774e7b887742d823d51f26"
format_stamp: "Formatted at 2026-09-23 15:06:27 on dist-test-slave-bb3x"
server_key: "fb8ca0f62b4cf5a86e557f927a67ad34"
server_key_iv: "426f7d68aeaf6a06cf0f6c5458602796"
server_key_version: "encryptionkey@0"
I20260923 15:06:27.138253 9069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:27.156260 9069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:27.158484 9069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:27.158644 9069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:27.158885 9069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:27.159215 9069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:27.159269 9069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:27.159334 9069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:27.159363 9069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:27.165130 9069 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:37457
I20260923 15:06:27.165207 9197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:37457 every 8 connection(s)
I20260923 15:06:27.165546 9069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:27.169217 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9069
I20260923 15:06:27.169330 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:27.169483 8169 external_mini_cluster.cc:1428] Setting key d1a68adc0166df82447f55b8504d871e
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175987, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:27.175299 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.166380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:37183 (local address 127.7.250.126:39195)
0923 15:06:27.166445 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.166447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.166621 (+ 174us) server_negotiation.cc:407] Connection header received
0923 15:06:27.167618 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.167619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.167634 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.167685 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:27.168342 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.169065 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.170517 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.170704 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.170758 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.172841 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:27.172843 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:27.172843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:27.172850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:27.174062 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.174492 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.174493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.174493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.174541 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.174770 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.174771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.174771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.174874 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:27.174922 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.175149 (+ 227us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.175210 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:06:27.175448 9200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.166337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39195 (local address 127.7.250.67:37183)
0923 15:06:27.166488 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.166539 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.167495 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.167756 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.167760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.168040 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:27.168235 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.168239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.169182 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.169183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.170460 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.170463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.170797 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.170798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.170899 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.171366 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:27.171374 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:27.172754 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:27.174131 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:27.174137 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:27.174138 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:27.174433 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:27.174596 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:27.174597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:27.174597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:27.174726 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:27.174927 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:27.174931 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:27.175042 (+ 111us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.175116 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.175233 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260923 15:06:27.176016 9198 heartbeater.cc:347] Connected to a master server at 127.7.250.126:39195
I20260923 15:06:27.176250 9198 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:27.176440 9198 heartbeater.cc:511] Master 127.7.250.126:39195 requested a full tablet report, sending...
I20260923 15:06:27.176833 8738 ts_manager.cc:194] Registered new tserver with Master: 90711cf27a774e7b887742d823d51f26 (127.7.250.67:37457)
I20260923 15:06:27.177174 8738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:37183
I20260923 15:06:27.181375 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:06:27.189617 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.183110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.7.250.126:39195)
0923 15:06:27.183211 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.183214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.183331 (+ 117us) server_negotiation.cc:407] Connection header received
0923 15:06:27.183410 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.183410 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.183420 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.183489 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:27.184043 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.184537 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.185242 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.185438 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.185524 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.186917 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:27.186918 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:27.186919 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:27.186927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:27.188234 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.188661 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.188662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.188663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.188744 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.188990 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.188991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.188992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.189204 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:27.189313 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.189430 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.189512 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 15:06:27.191892 8738 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:27.193526 8738 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:27.200343 9213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.196440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37457 (local address 127.0.0.1:35812)
0923 15:06:27.196821 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.196831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.196885 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.197055 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.197056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.197057 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.197089 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.197090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.198466 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.198467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.199367 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.199374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.200044 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.200045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.200123 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.200137 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.200207 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.200263 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":59,"threads_started":1}
I20260923 15:06:27.200352 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.196637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.7.250.67:37457)
0923 15:06:27.196817 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.196820 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.196875 (+ 55us) server_negotiation.cc:407] Connection header received
0923 15:06:27.196948 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.196950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.196966 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.197216 (+ 250us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.197307 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.198367 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.199453 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.199939 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.200044 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.200106 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.200231 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.200277 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260923 15:06:27.200569 9209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.196224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:34791 (local address 127.0.0.1:41348)
0923 15:06:27.196413 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.196467 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.196539 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.196895 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.196898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.196900 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.196975 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.196977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.198358 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.198359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.199409 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.199410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.200326 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.200327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.200392 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.200395 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.200428 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.200479 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":51,"threads_started":1}
I20260923 15:06:27.200793 9210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.196224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:35837 (local address 127.0.0.1:44360)
0923 15:06:27.196504 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.196514 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.196554 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.196926 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.196927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.196930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.196991 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.196992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.198069 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.198070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.199399 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.199402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.200375 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.200375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.200572 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.200576 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.200689 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.200728 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
I20260923 15:06:27.201077 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.196330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.7.250.66:35837)
0923 15:06:27.196551 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.196557 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.196569 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:27.196636 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.196638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.196738 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.196833 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.197086 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.197952 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.199505 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.200226 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.200296 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.200347 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.200957 (+ 610us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.200999 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":85,"threads_started":1}
I20260923 15:06:27.201077 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.196352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.7.250.65:34791)
0923 15:06:27.196691 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.196695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.196703 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:27.196761 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.196771 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.196786 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.197053 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.197134 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.198235 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.199505 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.200224 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.200867 (+ 643us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.200940 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.200965 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.201003 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":70,"threads_started":1}
I20260923 15:06:27.201086 9132 tablet_service.cc:1467] Processing CreateTablet for tablet 375013b1a5584385bbb374a5f470668f (DEFAULT_TABLE table=test-table [id=5234af5cd43e45ff8de7836302d9a892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:27.201525 9132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375013b1a5584385bbb374a5f470668f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:27.201697 8856 tablet_service.cc:1467] Processing CreateTablet for tablet 375013b1a5584385bbb374a5f470668f (DEFAULT_TABLE table=test-table [id=5234af5cd43e45ff8de7836302d9a892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:27.202003 8856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375013b1a5584385bbb374a5f470668f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:27.204818 8998 tablet_service.cc:1467] Processing CreateTablet for tablet 375013b1a5584385bbb374a5f470668f (DEFAULT_TABLE table=test-table [id=5234af5cd43e45ff8de7836302d9a892]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:27.205138 8998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 375013b1a5584385bbb374a5f470668f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:27.205173 9216 tablet_bootstrap.cc:492] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: Bootstrap starting.
I20260923 15:06:27.205365 9215 tablet_bootstrap.cc:492] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: Bootstrap starting.
I20260923 15:06:27.205778 9216 tablet_bootstrap.cc:654] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:27.205973 9215 tablet_bootstrap.cc:654] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:27.206503 9216 log.cc:824] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:27.206619 9215 log.cc:824] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:27.207320 9215 tablet_bootstrap.cc:492] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: No bootstrap required, opened a new log
I20260923 15:06:27.207355 9216 tablet_bootstrap.cc:492] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: No bootstrap required, opened a new log
I20260923 15:06:27.207412 9216 ts_tablet_manager.cc:1397] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:27.207410 9215 ts_tablet_manager.cc:1397] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:27.208650 9219 tablet_bootstrap.cc:492] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: Bootstrap starting.
I20260923 15:06:27.209233 9215 raft_consensus.cc:348] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.209381 9219 tablet_bootstrap.cc:654] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:27.209295 9216 raft_consensus.cc:348] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.209430 9215 raft_consensus.cc:374] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:27.209456 9216 raft_consensus.cc:374] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:27.209470 9215 raft_consensus.cc:729] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90711cf27a774e7b887742d823d51f26, State: Initialized, Role: FOLLOWER
I20260923 15:06:27.209504 9216 raft_consensus.cc:729] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 989917866f2946f98dd661d737d0d670, State: Initialized, Role: FOLLOWER
I20260923 15:06:27.209610 9216 consensus_queue.cc:260] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.209651 9215 consensus_queue.cc:260] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.209913 9216 ts_tablet_manager.cc:1428] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:06:27.209933 8930 heartbeater.cc:503] Master 127.7.250.126:39195 was elected leader, sending a full tablet report...
I20260923 15:06:27.209972 9215 ts_tablet_manager.cc:1428] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:27.209990 9219 log.cc:824] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:27.210091 9198 heartbeater.cc:503] Master 127.7.250.126:39195 was elected leader, sending a full tablet report...
I20260923 15:06:27.210798 9219 tablet_bootstrap.cc:492] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: No bootstrap required, opened a new log
I20260923 15:06:27.210868 9219 ts_tablet_manager.cc:1397] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:27.212509 9219 raft_consensus.cc:348] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.212639 9219 raft_consensus.cc:374] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:27.212694 9219 raft_consensus.cc:729] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdd60b35952419593b08bf1e477d594, State: Initialized, Role: FOLLOWER
I20260923 15:06:27.212821 9219 consensus_queue.cc:260] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.213094 9219 ts_tablet_manager.cc:1428] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:27.213111 9064 heartbeater.cc:503] Master 127.7.250.126:39195 was elected leader, sending a full tablet report...
W20260923 15:06:27.252940 9065 tablet.cc:2369] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:27.346971 8931 tablet.cc:2369] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:27.416184 9199 tablet.cc:2369] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:27.524626 9220 raft_consensus.cc:482] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:27.524842 9220 raft_consensus.cc:504] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.525249 9220 leader_election.cc:288] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90711cf27a774e7b887742d823d51f26 (127.7.250.67:37457), ccdd60b35952419593b08bf1e477d594 (127.7.250.66:35837)
I20260923 15:06:27.527662 9223 raft_consensus.cc:482] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:27.527809 9223 raft_consensus.cc:504] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.528124 9223 leader_election.cc:288] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 989917866f2946f98dd661d737d0d670 (127.7.250.65:34791), 90711cf27a774e7b887742d823d51f26 (127.7.250.67:37457)
I20260923 15:06:27.528842 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.525518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:50187 (local address 127.7.250.66:35837)
0923 15:06:27.525565 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.525567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.525684 (+ 117us) server_negotiation.cc:407] Connection header received
0923 15:06:27.525756 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.525757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.525768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.525839 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.526007 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.526579 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.527727 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.528251 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.528316 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.528326 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.528738 (+ 412us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.528778 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:06:27.528867 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.525441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:35837 (local address 127.7.250.65:50187)
0923 15:06:27.525596 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.525606 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.525689 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.525845 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.525846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.525848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.525928 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.525930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.526671 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.526672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.527622 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.527624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.528377 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.528378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.528640 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.528647 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.528726 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.528778 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":78,"thread_start_us":91,"threads_started":1}
I20260923 15:06:27.529237 9225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.525452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37457 (local address 127.7.250.65:45541)
0923 15:06:27.525608 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.525626 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.525703 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.525864 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.525864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.525867 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.525943 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.525945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.526695 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.526696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.527681 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.527684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.528643 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.528643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.528728 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.528733 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.528944 (+ 211us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.529063 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260923 15:06:27.529222 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375013b1a5584385bbb374a5f470668f" candidate_uuid: "989917866f2946f98dd661d737d0d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccdd60b35952419593b08bf1e477d594" is_pre_election: true
I20260923 15:06:27.529336 9018 raft_consensus.cc:2454] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989917866f2946f98dd661d737d0d670 in term 0.
I20260923 15:06:27.529533 8820 leader_election.cc:302] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 989917866f2946f98dd661d737d0d670, ccdd60b35952419593b08bf1e477d594; no voters:
I20260923 15:06:27.529664 9220 raft_consensus.cc:2781] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:27.529714 9220 raft_consensus.cc:482] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:27.529742 9220 raft_consensus.cc:3037] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:27.529907 9214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.525520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:45541 (local address 127.7.250.67:37457)
0923 15:06:27.525566 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.525569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.525692 (+ 123us) server_negotiation.cc:407] Connection header received
0923 15:06:27.525787 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.525788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.525796 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.525858 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.526016 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.526591 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.527819 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.528410 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.528478 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.528487 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.528814 (+ 327us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.529836 (+ 1022us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:06:27.530125 9152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375013b1a5584385bbb374a5f470668f" candidate_uuid: "989917866f2946f98dd661d737d0d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90711cf27a774e7b887742d823d51f26" is_pre_election: true
I20260923 15:06:27.530242 9152 raft_consensus.cc:2454] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 989917866f2946f98dd661d737d0d670 in term 0.
I20260923 15:06:27.530594 9220 raft_consensus.cc:504] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.530735 9220 leader_election.cc:288] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [CANDIDATE]: Term 1 election: Requested vote from peers 90711cf27a774e7b887742d823d51f26 (127.7.250.67:37457), ccdd60b35952419593b08bf1e477d594 (127.7.250.66:35837)
I20260923 15:06:27.530915 9018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375013b1a5584385bbb374a5f470668f" candidate_uuid: "989917866f2946f98dd661d737d0d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccdd60b35952419593b08bf1e477d594"
I20260923 15:06:27.531008 9018 raft_consensus.cc:3037] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:27.531394 9152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375013b1a5584385bbb374a5f470668f" candidate_uuid: "989917866f2946f98dd661d737d0d670" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90711cf27a774e7b887742d823d51f26"
I20260923 15:06:27.531468 9152 raft_consensus.cc:3037] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:27.531550 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.528333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:58491 (local address 127.7.250.65:34791)
0923 15:06:27.528522 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.528524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.528530 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:06:27.528549 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.528550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.528559 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.528607 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.528788 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.529347 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.530659 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.531180 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.531255 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.531271 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.531454 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.531495 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260923 15:06:27.531932 9018 raft_consensus.cc:2454] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989917866f2946f98dd661d737d0d670 in term 1.
I20260923 15:06:27.531944 9226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.528282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:34791 (local address 127.7.250.66:58491)
0923 15:06:27.528393 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.528403 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.528458 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.528625 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.528626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.528628 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.528702 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.528703 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.529440 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.529440 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.530580 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.530584 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.531268 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.531269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.531359 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.531363 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.531792 (+ 429us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.531844 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
I20260923 15:06:27.532119 8820 leader_election.cc:302] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 989917866f2946f98dd661d737d0d670, ccdd60b35952419593b08bf1e477d594; no voters:
I20260923 15:06:27.532198 9220 raft_consensus.cc:2781] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:27.532322 9152 raft_consensus.cc:2454] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 989917866f2946f98dd661d737d0d670 in term 1.
I20260923 15:06:27.532338 9220 raft_consensus.cc:686] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [term 1 LEADER]: Becoming Leader. State: Replica: 989917866f2946f98dd661d737d0d670, State: Running, Role: LEADER
I20260923 15:06:27.532434 9220 consensus_queue.cc:237] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } }
I20260923 15:06:27.532475 8884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375013b1a5584385bbb374a5f470668f" candidate_uuid: "ccdd60b35952419593b08bf1e477d594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "989917866f2946f98dd661d737d0d670" is_pre_election: true
I20260923 15:06:27.532951 9227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.528438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37457 (local address 127.7.250.66:43057)
0923 15:06:27.528583 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:27.528593 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:27.528641 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:27.529715 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:27.529716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:27.529718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:27.529785 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.529787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.530657 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.530658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.531753 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:27.531757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.532634 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.532635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:27.532761 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.532766 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:27.532827 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:06:27.532874 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":47,"threads_started":1}
I20260923 15:06:27.533176 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.528482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:43057 (local address 127.7.250.67:37457)
0923 15:06:27.529500 (+ 1018us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.529502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.529510 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:27.529535 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.529536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.529543 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.529697 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:27.529874 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.530563 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.531864 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.532452 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.532533 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.532541 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.533061 (+ 520us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.533110 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":90,"threads_started":1}
I20260923 15:06:27.533233 8738 catalog_manager.cc:5495] T 375013b1a5584385bbb374a5f470668f P 989917866f2946f98dd661d737d0d670 reported cstate change: term changed from 0 to 1, leader changed from <none> to 989917866f2946f98dd661d737d0d670 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "989917866f2946f98dd661d737d0d670" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "989917866f2946f98dd661d737d0d670" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34791 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90711cf27a774e7b887742d823d51f26" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccdd60b35952419593b08bf1e477d594" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35837 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:27.533736 9152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "375013b1a5584385bbb374a5f470668f" candidate_uuid: "ccdd60b35952419593b08bf1e477d594" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90711cf27a774e7b887742d823d51f26" is_pre_election: true
I20260923 15:06:27.533860 9152 raft_consensus.cc:2379] T 375013b1a5584385bbb374a5f470668f P 90711cf27a774e7b887742d823d51f26 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ccdd60b35952419593b08bf1e477d594 in current term 1: Already voted for candidate 989917866f2946f98dd661d737d0d670 in this term.
I20260923 15:06:27.534052 8951 leader_election.cc:302] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ccdd60b35952419593b08bf1e477d594; no voters: 90711cf27a774e7b887742d823d51f26, 989917866f2946f98dd661d737d0d670
I20260923 15:06:27.534221 9223 raft_consensus.cc:2726] T 375013b1a5584385bbb374a5f470668f P ccdd60b35952419593b08bf1e477d594 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175986, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:27.558156 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.550803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41350 (local address 127.7.250.65:34791)
0923 15:06:27.550869 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.550872 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.551002 (+ 130us) server_negotiation.cc:407] Connection header received
0923 15:06:27.551048 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.551049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.551058 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.551107 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:27.551680 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.552153 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.552862 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.553054 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.553109 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.555415 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:27.555427 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:27.555427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:27.555434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:27.557130 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.557478 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.557481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.557481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.557535 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.557699 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.557699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.557700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.557831 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:27.557942 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.557990 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.558048 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175987, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175987, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:27.580431 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.573526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.7.250.65:34791)
0923 15:06:27.573609 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.573616 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.573661 (+ 45us) server_negotiation.cc:407] Connection header received
0923 15:06:27.573723 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.573724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.573733 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.573781 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:27.574407 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.574891 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.575567 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.575786 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.575842 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.577943 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:27.577944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:27.577944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:27.577950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:27.579372 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.579661 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.579661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.579662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.579709 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.579919 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.579920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.579921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.580029 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:27.580146 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.580250 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.580312 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[8694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175987, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:27.602795 9212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:27.595796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41362 (local address 127.7.250.65:34791)
0923 15:06:27.595894 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:27.595896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:27.595946 (+ 50us) server_negotiation.cc:407] Connection header received
0923 15:06:27.595996 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:27.595997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:27.596006 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:27.596082 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:27.596636 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.597115 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.597831 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:27.598008 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:27.598062 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:27.600296 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:27.600297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:27.600297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:27.600304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:27.601563 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.601882 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.601883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.601883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.601934 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:27.602147 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:27.602148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:27.602148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:27.602235 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:27.602479 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:27.602564 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 15:06:27.602625 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260923 15:06:27.603403 8844 tablet_service.cc:2908] Rejecting scan request for tablet 375013b1a5584385bbb374a5f470668f: Uninitialized: clean time has not yet been initialized
I20260923 15:06:27.604303 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8801
I20260923 15:06:27.609457 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8935
I20260923 15:06:27.613977 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9069
I20260923 15:06:27.618422 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 8707
2026-09-23T15:06:27Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2290 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[9243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[9243](info): set up 2 sockets
Sep 23 15:06:27 dist-test-slave-bb3x krb5kdc[9243](info): commencing operation
krb5kdc: starting...
W20260923 15:06:28.655458 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:28 dist-test-slave-bb3x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:28.777813 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42681
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:42681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:28.859305 9256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:28.859544 9256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:28.859583 9256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:28.860756 9256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:28.860822 9256 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 15:06:28.860846 9256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:28.860879 9256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:28.860905 9256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:28.860935 9256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:28.862497 9256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:42681
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:28.862782 9256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:28.863008 9256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:28.868021 9264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:28.868031 9262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:28.868100 9256 server_base.cc:1034] running on GCE node
W20260923 15:06:28.868029 9261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:28.868513 9256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:28.868813 9256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:28.869984 9256 hybrid_clock.cc:648] HybridClock initialized: now 1790175988869972 us; error 42 us; skew 500 ppm
Sep 23 15:06:28 dist-test-slave-bb3x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175988, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:28.872610 9256 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:28.873059 9256 webserver.cc:460] Webserver started at http://127.7.250.126:39961/ using document root <none> and password file <none>
I20260923 15:06:28.873337 9256 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:28.873407 9256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:28.873564 9256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:28.874702 9256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "fdaf64a820474395ad9d4ca0e75f8de4"
format_stamp: "Formatted at 2026-09-23 15:06:28 on dist-test-slave-bb3x"
server_key: "1593b62c536bbd14bb323eb85a33caae"
server_key_iv: "ad92178b61747c0d17d7ca97ae1c01c0"
server_key_version: "encryptionkey@0"
I20260923 15:06:28.875062 9256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "fdaf64a820474395ad9d4ca0e75f8de4"
format_stamp: "Formatted at 2026-09-23 15:06:28 on dist-test-slave-bb3x"
server_key: "1593b62c536bbd14bb323eb85a33caae"
server_key_iv: "ad92178b61747c0d17d7ca97ae1c01c0"
server_key_version: "encryptionkey@0"
I20260923 15:06:28.876443 9256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:28.877279 9271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:28.877477 9256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:28.877569 9256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "fdaf64a820474395ad9d4ca0e75f8de4"
format_stamp: "Formatted at 2026-09-23 15:06:28 on dist-test-slave-bb3x"
server_key: "1593b62c536bbd14bb323eb85a33caae"
server_key_iv: "ad92178b61747c0d17d7ca97ae1c01c0"
server_key_version: "encryptionkey@0"
I20260923 15:06:28.877652 9256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:28.905323 9256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:28.907586 9256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:28.907769 9256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:28.912465 9256 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:42681
I20260923 15:06:28.912539 9333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:42681 every 8 connection(s)
I20260923 15:06:28.912846 9256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:28.913110 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9256
I20260923 15:06:28.913251 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:28.913465 8169 external_mini_cluster.cc:1428] Setting key 3fb99c067941973e911814927019e084
I20260923 15:06:28.913578 9334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:28.917007 9334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: Bootstrap starting.
I20260923 15:06:28.917639 9334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:28.917935 9334 log.cc:824] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:28.918793 9334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: No bootstrap required, opened a new log
Sep 23 15:06:28 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:28.920632 9334 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42681 } }
I20260923 15:06:28.920794 9334 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:28.920861 9334 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdaf64a820474395ad9d4ca0e75f8de4, State: Initialized, Role: FOLLOWER
I20260923 15:06:28.920992 9334 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42681 } }
I20260923 15:06:28.921069 9334 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:28.921092 9334 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:28.921195 9334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:28.921941 9334 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42681 } }
I20260923 15:06:28.922065 9334 leader_election.cc:302] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdaf64a820474395ad9d4ca0e75f8de4; no voters:
I20260923 15:06:28.922232 9334 leader_election.cc:288] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:28.922312 9339 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:28.922457 9339 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [term 1 LEADER]: Becoming Leader. State: Replica: fdaf64a820474395ad9d4ca0e75f8de4, State: Running, Role: LEADER
I20260923 15:06:28.922520 9334 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:28.922636 9339 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42681 } }
I20260923 15:06:28.923080 9341 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdaf64a820474395ad9d4ca0e75f8de4. Latest consensus state: current_term: 1 leader_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42681 } } }
I20260923 15:06:28.923174 9341 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:28.924477 9355 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:28.924532 9355 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:28.924608 9348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:28.923902 9340 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdaf64a820474395ad9d4ca0e75f8de4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42681 } } }
I20260923 15:06:28.924803 9340 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:28.924854 9348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:28.925494 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:28.913957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39532 (local address 127.7.250.126:42681)
0923 15:06:28.914149 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:28.914159 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:28.914201 (+ 42us) server_negotiation.cc:407] Connection header received
0923 15:06:28.914834 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:28.914843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:28.914908 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:28.915066 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:28.915630 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:28.916983 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:28.918012 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:28.918605 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:28.918717 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:28.921396 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:28.921400 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:28.921400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:28.921412 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:28.923500 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:28.923956 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:28.923959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:28.923960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:28.924089 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:28.924388 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:28.924389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:28.924390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:28.924722 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:28.924853 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:28.925117 (+ 264us) server_negotiation.cc:299] Negotiation successful
0923 15:06:28.925256 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":91,"threads_started":1}
I20260923 15:06:28.926433 9348 catalog_manager.cc:1347] Generated new cluster ID: 25af0a4940e34feebf50e6ec18fe0cde
I20260923 15:06:28.926474 9348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:28.939833 9348 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:28.940397 9348 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:28.950296 9348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: Generated new TSK 0
I20260923 15:06:28.950500 9348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:28.977586 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42681
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:29.059705 9360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:29.059940 9360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:29.059966 9360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:29.061151 9360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:29.061198 9360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:29.061219 9360 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:29.061237 9360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:29.062760 9360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:29.063014 9360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:29.063262 9360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:29.068583 9366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:29.068583 9368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:29.068737 9360 server_base.cc:1034] running on GCE node
W20260923 15:06:29.068800 9365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:29.069089 9360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:29.069402 9360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:29.070576 9360 hybrid_clock.cc:648] HybridClock initialized: now 1790175989070552 us; error 41 us; skew 500 ppm
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:29.073315 9360 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:29.073757 9360 webserver.cc:460] Webserver started at http://127.7.250.65:39769/ using document root <none> and password file <none>
I20260923 15:06:29.074005 9360 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:29.074075 9360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:29.074223 9360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:29.075317 9360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "5d06efda57324d60b34e6ff4eddbdd3b"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "8209f649d0f11d80f40c05fc83388b1a"
server_key_iv: "e0e6bff6248b5b154b2e92888dba8762"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.075692 9360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "5d06efda57324d60b34e6ff4eddbdd3b"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "8209f649d0f11d80f40c05fc83388b1a"
server_key_iv: "e0e6bff6248b5b154b2e92888dba8762"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.077061 9360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:29.077826 9375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.077991 9360 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:29.078097 9360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "5d06efda57324d60b34e6ff4eddbdd3b"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "8209f649d0f11d80f40c05fc83388b1a"
server_key_iv: "e0e6bff6248b5b154b2e92888dba8762"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.078172 9360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:29.087124 9360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:29.089411 9360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:29.089581 9360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:29.090198 9360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:29.090317 9360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.090381 9360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:29.090435 9360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:29.098826 9360 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:44887
I20260923 15:06:29.099298 9360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:29.100443 9490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:44887 every 8 connection(s)
I20260923 15:06:29.102713 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.090638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:42743 (local address 127.7.250.126:42681)
0923 15:06:29.090711 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.090714 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.091013 (+ 299us) server_negotiation.cc:407] Connection header received
0923 15:06:29.091798 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.091800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.091814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.091885 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.092776 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.093620 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.095167 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.095619 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.095681 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.098675 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.098676 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.098677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.098724 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.100471 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.101604 (+ 1133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.101605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.101606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.101671 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.102009 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.102009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.102010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.102141 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.102217 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.102517 (+ 300us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.102609 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:06:29.102893 9384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.090623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.7.250.65:42743)
0923 15:06:29.090831 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.090900 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.091671 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.091977 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.091981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.092375 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.092651 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.092656 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.093739 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.093740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.095057 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.095060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.095778 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.095779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.095931 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.096527 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.096539 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.098542 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.100572 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.100579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.100582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.100914 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.101774 (+ 860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.101775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.101775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.101929 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.102210 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.102217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.102385 (+ 168us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.102498 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.102659 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260923 15:06:29.103446 9491 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42681
I20260923 15:06:29.103528 9491 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:29.103729 9491 heartbeater.cc:511] Master 127.7.250.126:42681 requested a full tablet report, sending...
I20260923 15:06:29.104282 9288 ts_manager.cc:194] Registered new tserver with Master: 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65:44887)
I20260923 15:06:29.105048 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9360
I20260923 15:06:29.105157 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:29.105209 9288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:42743
I20260923 15:06:29.105315 8169 external_mini_cluster.cc:1428] Setting key a823dc63fadb37aade262fd6a912a130
I20260923 15:06:29.110743 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.104641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:46033 (local address 127.7.250.126:42681)
0923 15:06:29.104680 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.104682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.104763 (+ 81us) server_negotiation.cc:407] Connection header received
0923 15:06:29.104822 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.104823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.104830 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.104871 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.105242 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.105954 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.106873 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.107080 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.107162 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.108083 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.108084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.108084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.108093 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.109502 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.109908 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.109909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.109909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.109954 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.110217 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.110218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.110218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.110379 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.110431 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.110621 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.110682 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:29.110857 9497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.104552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.7.250.65:46033)
0923 15:06:29.104700 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.104711 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.104769 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.104883 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.104884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.105074 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.105152 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.105153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.106069 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.106070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.106771 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.106773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.107163 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.107164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.107246 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.107696 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.107701 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.107994 (+ 293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.109590 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.109591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.109591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.109829 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.110018 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.110019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.110019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.110126 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.110447 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.110448 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.110506 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.110592 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.110710 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:29.134107 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42681
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:29.215276 9500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:29.215473 9500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:29.215497 9500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:29.216698 9500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:29.216753 9500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:29.216774 9500 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:29.216792 9500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:29.218299 9500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:29.218544 9500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:29.218806 9500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:29.223934 9508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:29.224027 9500 server_base.cc:1034] running on GCE node
W20260923 15:06:29.223943 9505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:29.223943 9506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:29.224373 9500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:29.224579 9500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:29.225754 9500 hybrid_clock.cc:648] HybridClock initialized: now 1790175989225733 us; error 47 us; skew 500 ppm
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:29.228364 9500 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:29.228879 9500 webserver.cc:460] Webserver started at http://127.7.250.66:35719/ using document root <none> and password file <none>
I20260923 15:06:29.229094 9500 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:29.229137 9500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:29.229228 9500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:29.230237 9500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "a62deb54bc1547a1bceaf2630ba40bbc"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "8485253800bfb616e9c26e92501cffd3"
server_key_iv: "872a3a625e0d0f37bde11d8eb2589b86"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.230639 9500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "a62deb54bc1547a1bceaf2630ba40bbc"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "8485253800bfb616e9c26e92501cffd3"
server_key_iv: "872a3a625e0d0f37bde11d8eb2589b86"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.231917 9500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:29.232676 9515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.232841 9500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:29.232928 9500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "a62deb54bc1547a1bceaf2630ba40bbc"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "8485253800bfb616e9c26e92501cffd3"
server_key_iv: "872a3a625e0d0f37bde11d8eb2589b86"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.232983 9500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:29.253305 9500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:29.256525 9500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:29.256755 9500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:29.257491 9500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:29.257567 9500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.257611 9500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:29.257638 9500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.269635 9500 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:41375
I20260923 15:06:29.269876 9631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:41375 every 8 connection(s)
I20260923 15:06:29.270109 9500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:29.275462 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.258046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:43601 (local address 127.7.250.126:42681)
0923 15:06:29.258098 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.258101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.258678 (+ 577us) server_negotiation.cc:407] Connection header received
0923 15:06:29.259577 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.259579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.259615 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.259737 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.260854 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.261917 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.267805 (+ 5888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.268358 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.268454 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.271943 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.271945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.271945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.271954 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.273842 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.274391 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.274394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.274394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.274460 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.274740 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.274741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.274741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.274876 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.274962 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.275270 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.275360 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:06:29.275702 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.258305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.7.250.66:43601)
0923 15:06:29.258522 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.258581 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.259336 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.259912 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.259917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.260329 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.260695 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.260703 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.262043 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.262044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.267638 (+ 5594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.267643 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.268488 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.268489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.268690 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.269566 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.269578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.271820 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.273940 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.273947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.273950 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.274245 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.274537 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.274538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.274539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.274655 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.274953 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.274960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.275118 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.275234 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.275415 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260923 15:06:29.276191 9632 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42681
I20260923 15:06:29.276288 9632 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:29.276510 9632 heartbeater.cc:511] Master 127.7.250.126:42681 requested a full tablet report, sending...
I20260923 15:06:29.276919 9287 ts_manager.cc:194] Registered new tserver with Master: a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375)
I20260923 15:06:29.277527 9287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:43601
I20260923 15:06:29.279201 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9500
I20260923 15:06:29.279289 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:29.279477 8169 external_mini_cluster.cc:1428] Setting key aeaf0f122a959c3cc3e844b87a36d5f9
I20260923 15:06:29.283843 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.277304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.7.250.66:40467)
0923 15:06:29.277442 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.277452 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.277515 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.277639 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.277640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.277869 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.277947 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.277949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.278725 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.278726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.279411 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.279414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.279933 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.279933 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.279990 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.280405 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.280411 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.280826 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.282726 (+ 1900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.282727 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.282728 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.282898 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.283102 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.283103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.283103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.283161 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.283571 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.283573 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.283628 (+ 55us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.283694 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.283771 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260923 15:06:29.283985 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.277378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:40467 (local address 127.7.250.126:42681)
0923 15:06:29.277438 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.277440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.277506 (+ 66us) server_negotiation.cc:407] Connection header received
0923 15:06:29.277557 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.277559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.277569 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.277633 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.278022 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.278638 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.279514 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.279823 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.279870 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.280923 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.280925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.280925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.280932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.282629 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.282980 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.282981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.282982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.283037 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.283235 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.283235 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.283236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.283463 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.283538 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.283770 (+ 232us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.283873 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:29.308813 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42681
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:29.399226 9641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:29.399448 9641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:29.399475 9641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:29.400755 9641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:29.400797 9641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:29.400818 9641 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:29.400835 9641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:29.402364 9641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:42681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:29.402614 9641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:29.402892 9641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:29.408141 9647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:29.408142 9646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:29.408145 9649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:29.408504 9641 server_base.cc:1034] running on GCE node
I20260923 15:06:29.408739 9641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:29.408984 9641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:29.410135 9641 hybrid_clock.cc:648] HybridClock initialized: now 1790175989410114 us; error 36 us; skew 500 ppm
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:29.412844 9641 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:29.413234 9641 webserver.cc:460] Webserver started at http://127.7.250.67:45271/ using document root <none> and password file <none>
I20260923 15:06:29.413439 9641 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:29.413479 9641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:29.413565 9641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:29.414666 9641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "a4cbff87688e409cb8a28d140cc885b0"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "95bad0c677b6240396081928bf8dde39"
server_key_iv: "aa570fc4aa88cabd4d516c590f6ae653"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.414955 9641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "a4cbff87688e409cb8a28d140cc885b0"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "95bad0c677b6240396081928bf8dde39"
server_key_iv: "aa570fc4aa88cabd4d516c590f6ae653"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.416251 9641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:29.416971 9656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.417164 9641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:29.417260 9641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "a4cbff87688e409cb8a28d140cc885b0"
format_stamp: "Formatted at 2026-09-23 15:06:29 on dist-test-slave-bb3x"
server_key: "95bad0c677b6240396081928bf8dde39"
server_key_iv: "aa570fc4aa88cabd4d516c590f6ae653"
server_key_version: "encryptionkey@0"
I20260923 15:06:29.417309 9641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:29.428196 9641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:29.430217 9641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:29.430415 9641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:29.430965 9641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:29.431010 9641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.431031 9641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:29.431044 9641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:29.438119 9641 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:42025
I20260923 15:06:29.438191 9771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:42025 every 8 connection(s)
I20260923 15:06:29.438506 9641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:29.441356 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.431466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:55183 (local address 127.7.250.126:42681)
0923 15:06:29.431538 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.431541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.431712 (+ 171us) server_negotiation.cc:407] Connection header received
0923 15:06:29.432192 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.432194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.432209 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.432288 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.433231 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.434071 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.435261 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.435569 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.435675 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.438381 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.438383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.438383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.438391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.439969 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.440384 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.440386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.440386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.440448 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.440699 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.440700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.440701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.440825 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.440904 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.441173 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.441258 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:06:29.441552 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.431428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.7.250.67:55183)
0923 15:06:29.431577 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.431616 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.432091 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.432404 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.432413 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.432823 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.433121 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.433127 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.434195 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.434196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.435165 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.435167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.435751 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.435752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.435858 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.436344 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.436357 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.438293 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.440072 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.440078 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.440080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.440306 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.440517 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.440518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.440518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.440629 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.440893 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.440900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.441041 (+ 141us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.441172 (+ 131us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.441332 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
I20260923 15:06:29.442098 9772 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42681
I20260923 15:06:29.442184 9772 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:29.442397 9772 heartbeater.cc:511] Master 127.7.250.126:42681 requested a full tablet report, sending...
I20260923 15:06:29.442797 9287 ts_manager.cc:194] Registered new tserver with Master: a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025)
I20260923 15:06:29.443302 9287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:55183
I20260923 15:06:29.444751 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9641
I20260923 15:06:29.444854 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:29.445007 8169 external_mini_cluster.cc:1428] Setting key bf90faec5d9c0e29bc22330295a7f413
I20260923 15:06:29.445779 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:06:29.449478 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.443180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.7.250.67:51929)
0923 15:06:29.443343 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.443354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.443420 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.443556 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.443557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.443785 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.443852 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.443854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.444508 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.444509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.445204 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.445207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.445645 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.445646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.445724 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.446203 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.446208 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.446579 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.448390 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.448391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.448392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.448558 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.448833 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.448834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.448834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.448894 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.449219 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.449220 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.449261 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.449323 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.449402 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260923 15:06:29.449888 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.443240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:51929 (local address 127.7.250.126:42681)
0923 15:06:29.443280 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.443282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.443428 (+ 146us) server_negotiation.cc:407] Connection header received
0923 15:06:29.443495 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.443496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.443506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.443552 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.443932 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.444405 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.445301 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.445508 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.445565 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.446675 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.446677 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.446677 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.446683 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.448302 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.448676 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.448677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.448678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.448739 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.448992 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.448993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.448993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.449136 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.449210 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.449366 (+ 156us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.449448 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:29.468565 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.461852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39544 (local address 127.7.250.126:42681)
0923 15:06:29.461918 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.461920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.461928 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:29.461953 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.461953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.461963 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.462028 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.462646 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.463149 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.463849 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.464025 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.464075 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.466183 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.466185 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.466185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.466194 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.467630 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.467962 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.467963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.467963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.468015 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.468174 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.468174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.468174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.468281 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.468358 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.468438 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.468496 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:06:29.470598 9287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:29.472198 9287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:29.479177 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.475029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.7.250.66:41375)
0923 15:06:29.475240 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.475244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.475342 (+ 98us) server_negotiation.cc:407] Connection header received
0923 15:06:29.475451 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.475454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.475469 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.475521 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.475727 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.476519 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478115 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.478706 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478802 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.478863 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.479026 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.479069 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260923 15:06:29.479173 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.474622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42025 (local address 127.0.0.1:55164)
0923 15:06:29.474896 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.474939 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.475010 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.475385 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.475386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.475388 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.475446 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.475448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.476722 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.476723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.477856 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.477859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.478909 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.478974 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.478979 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.479027 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.479074 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":83,"thread_start_us":60,"threads_started":1}
I20260923 15:06:29.479177 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.474729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:44887 (local address 127.0.0.1:52126)
0923 15:06:29.474982 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.474991 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.475043 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.475260 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.475263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.475266 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.475337 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.475340 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.476664 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.476664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.477983 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.477986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.478926 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.478990 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.478994 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.479042 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.479075 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":76,"thread_start_us":89,"threads_started":1}
I20260923 15:06:29.479379 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.474830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.7.250.65:44887)
0923 15:06:29.474991 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.474995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.475034 (+ 39us) server_negotiation.cc:407] Connection header received
0923 15:06:29.475112 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.475114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.475130 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.475417 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.475476 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.476586 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478080 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.478826 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478921 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.478971 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.479218 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.479262 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260923 15:06:29.479177 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.474829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41375 (local address 127.0.0.1:43054)
0923 15:06:29.475289 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.475298 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.475366 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.475604 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.475604 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.475605 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.475640 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.475641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.476596 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.476597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.478021 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.478023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.478808 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.478889 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.478902 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.478999 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.479061 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":67,"thread_start_us":62,"threads_started":1}
I20260923 15:06:29.479580 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.474753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55164 (local address 127.7.250.67:42025)
0923 15:06:29.475171 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.475176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.475186 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:29.475225 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.475227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.475242 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.475321 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.475511 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.476633 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.477943 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.478739 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.478811 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.478870 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.479304 (+ 434us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.479359 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":88,"threads_started":1}
I20260923 15:06:29.480063 9416 tablet_service.cc:1467] Processing CreateTablet for tablet f2eb6236ea7f407399f6ccd608d5107a (DEFAULT_TABLE table=test-table [id=37190efa0ba94d66a50f8b090ddffcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:29.480134 9706 tablet_service.cc:1467] Processing CreateTablet for tablet f2eb6236ea7f407399f6ccd608d5107a (DEFAULT_TABLE table=test-table [id=37190efa0ba94d66a50f8b090ddffcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:29.480440 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2eb6236ea7f407399f6ccd608d5107a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.480466 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2eb6236ea7f407399f6ccd608d5107a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.480600 9566 tablet_service.cc:1467] Processing CreateTablet for tablet f2eb6236ea7f407399f6ccd608d5107a (DEFAULT_TABLE table=test-table [id=37190efa0ba94d66a50f8b090ddffcf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:29.480931 9566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f2eb6236ea7f407399f6ccd608d5107a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.483893 9795 tablet_bootstrap.cc:492] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Bootstrap starting.
I20260923 15:06:29.484187 9797 tablet_bootstrap.cc:492] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Bootstrap starting.
I20260923 15:06:29.484256 9796 tablet_bootstrap.cc:492] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Bootstrap starting.
I20260923 15:06:29.484807 9796 tablet_bootstrap.cc:654] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.484892 9797 tablet_bootstrap.cc:654] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.485113 9795 tablet_bootstrap.cc:654] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.485152 9796 log.cc:824] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:29.485230 9797 log.cc:824] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:29.485459 9795 log.cc:824] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:29.486054 9796 tablet_bootstrap.cc:492] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: No bootstrap required, opened a new log
I20260923 15:06:29.486127 9796 ts_tablet_manager.cc:1397] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:29.486160 9797 tablet_bootstrap.cc:492] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: No bootstrap required, opened a new log
I20260923 15:06:29.486169 9795 tablet_bootstrap.cc:492] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: No bootstrap required, opened a new log
I20260923 15:06:29.486229 9797 ts_tablet_manager.cc:1397] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:29.486229 9795 ts_tablet_manager.cc:1397] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:29.487574 9796 raft_consensus.cc:348] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.487744 9796 raft_consensus.cc:374] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.487790 9796 raft_consensus.cc:729] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62deb54bc1547a1bceaf2630ba40bbc, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.487905 9796 consensus_queue.cc:260] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.488189 9796 ts_tablet_manager.cc:1428] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:29.488216 9632 heartbeater.cc:503] Master 127.7.250.126:42681 was elected leader, sending a full tablet report...
I20260923 15:06:29.488266 9797 raft_consensus.cc:348] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.488241 9795 raft_consensus.cc:348] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.488389 9795 raft_consensus.cc:374] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.488389 9797 raft_consensus.cc:374] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.488422 9797 raft_consensus.cc:729] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d06efda57324d60b34e6ff4eddbdd3b, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.488422 9795 raft_consensus.cc:729] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4cbff87688e409cb8a28d140cc885b0, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.488528 9797 consensus_queue.cc:260] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.488570 9795 consensus_queue.cc:260] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.488814 9795 ts_tablet_manager.cc:1428] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:29.488837 9772 heartbeater.cc:503] Master 127.7.250.126:42681 was elected leader, sending a full tablet report...
I20260923 15:06:29.488986 9491 heartbeater.cc:503] Master 127.7.250.126:42681 was elected leader, sending a full tablet report...
I20260923 15:06:29.489064 9797 ts_tablet_manager.cc:1428] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
W20260923 15:06:29.520877 9633 tablet.cc:2369] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:29.600642 9492 tablet.cc:2369] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:29.623170 9802 raft_consensus.cc:482] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:29.623374 9802 raft_consensus.cc:504] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.623798 9802 leader_election.cc:288] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025), a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375)
I20260923 15:06:29.626915 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.624112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:45725 (local address 127.7.250.67:42025)
0923 15:06:29.624192 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.624195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.624211 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:06:29.624296 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.624298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.624308 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.624377 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.624595 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.625103 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.626099 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.626544 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.626638 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.626653 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.626802 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.626840 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 15:06:29.626915 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.624324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:35759 (local address 127.7.250.66:41375)
0923 15:06:29.624378 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.624381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.624391 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:29.624420 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.624421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.624431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.624481 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.624640 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.625112 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.626097 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.626554 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.626637 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.626646 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.626808 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.626848 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:06:29.626937 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.624226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41375 (local address 127.7.250.65:35759)
0923 15:06:29.624327 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.624335 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.624373 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.624485 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.624486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.624488 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.624560 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.624561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.625207 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.625208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.626002 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.626004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.626637 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.626638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.626732 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.626736 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.626802 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.626843 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":50,"threads_started":1}
I20260923 15:06:29.626943 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.624003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42025 (local address 127.7.250.65:45725)
0923 15:06:29.624157 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.624169 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.624223 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.624438 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.624439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.624442 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.624511 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.624513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.625211 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.625212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.626007 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.626009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.626638 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.626639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.626712 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.626722 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.626795 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.626839 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":42,"thread_start_us":102,"threads_started":1}
I20260923 15:06:29.627324 9586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2eb6236ea7f407399f6ccd608d5107a" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" is_pre_election: true
I20260923 15:06:29.627503 9586 raft_consensus.cc:2454] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 0.
I20260923 15:06:29.627494 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2eb6236ea7f407399f6ccd608d5107a" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4cbff87688e409cb8a28d140cc885b0" is_pre_election: true
I20260923 15:06:29.627669 9726 raft_consensus.cc:2454] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 0.
I20260923 15:06:29.627735 9378 leader_election.cc:302] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d06efda57324d60b34e6ff4eddbdd3b, a62deb54bc1547a1bceaf2630ba40bbc; no voters:
I20260923 15:06:29.627940 9802 raft_consensus.cc:2781] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:29.627996 9802 raft_consensus.cc:482] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:29.628032 9802 raft_consensus.cc:3037] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.628752 9802 raft_consensus.cc:504] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.628898 9802 leader_election.cc:288] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 election: Requested vote from peers a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025), a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375)
I20260923 15:06:29.629076 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2eb6236ea7f407399f6ccd608d5107a" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4cbff87688e409cb8a28d140cc885b0"
I20260923 15:06:29.629082 9586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f2eb6236ea7f407399f6ccd608d5107a" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62deb54bc1547a1bceaf2630ba40bbc"
I20260923 15:06:29.629174 9726 raft_consensus.cc:3037] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.629172 9586 raft_consensus.cc:3037] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.629868 9586 raft_consensus.cc:2454] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 1.
I20260923 15:06:29.629933 9726 raft_consensus.cc:2454] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 1.
I20260923 15:06:29.630028 9378 leader_election.cc:302] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d06efda57324d60b34e6ff4eddbdd3b, a62deb54bc1547a1bceaf2630ba40bbc; no voters:
I20260923 15:06:29.630110 9802 raft_consensus.cc:2781] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:29.630219 9802 raft_consensus.cc:686] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 LEADER]: Becoming Leader. State: Replica: 5d06efda57324d60b34e6ff4eddbdd3b, State: Running, Role: LEADER
I20260923 15:06:29.630352 9802 consensus_queue.cc:237] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } }
I20260923 15:06:29.631201 9288 catalog_manager.cc:5495] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65). New cstate: current_term: 1 leader_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } health_report { overall_health: HEALTHY } } }
I20260923 15:06:29.660586 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.657939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.7.250.65:44887)
0923 15:06:29.658005 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.658007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.658017 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:29.658083 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.658084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.658110 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.658194 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:29.658380 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.658851 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.659737 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.659936 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.660008 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.660233 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.660334 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.660418 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.660469 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.660491 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:06:29.662510 9726 raft_consensus.cc:1261] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Refusing update from remote peer 5d06efda57324d60b34e6ff4eddbdd3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:29.662524 9586 raft_consensus.cc:1261] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Refusing update from remote peer 5d06efda57324d60b34e6ff4eddbdd3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:29.662869 9806 consensus_queue.cc:1035] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:29.663009 9802 consensus_queue.cc:1035] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:29.667943 9809 mvcc.cc:205] Tried to move back new op lower bound from 7332560853654781952 to 7332560853526634496. Current Snapshot: MvccSnapshot[applied={T|T < 7332560853654781952}]
I20260923 15:06:29.668285 9810 mvcc.cc:205] Tried to move back new op lower bound from 7332560853654781952 to 7332560853526634496. Current Snapshot: MvccSnapshot[applied={T|T < 7332560853654781952}]
I20260923 15:06:29.668577 9812 mvcc.cc:205] Tried to move back new op lower bound from 7332560853654781952 to 7332560853526634496. Current Snapshot: MvccSnapshot[applied={T|T < 7332560853654781952}]
I20260923 15:06:29.672155 9287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:29.672261 9287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:29.673113 9287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: Sending DeleteTablet for 3 replicas of tablet f2eb6236ea7f407399f6ccd608d5107a
I20260923 15:06:29.673496 9706 tablet_service.cc:1514] Processing DeleteTablet for tablet f2eb6236ea7f407399f6ccd608d5107a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:29 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:55164
I20260923 15:06:29.673635 9566 tablet_service.cc:1514] Processing DeleteTablet for tablet f2eb6236ea7f407399f6ccd608d5107a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:29 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:43054
I20260923 15:06:29.673810 9416 tablet_service.cc:1514] Processing DeleteTablet for tablet f2eb6236ea7f407399f6ccd608d5107a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:29 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:52126
I20260923 15:06:29.673868 9818 tablet_replica.cc:330] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: stopping tablet replica
I20260923 15:06:29.673966 9287 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:29.674109 9287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:29.674275 9819 tablet_replica.cc:330] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: stopping tablet replica
I20260923 15:06:29.674052 9818 raft_consensus.cc:2229] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:29.674407 9819 raft_consensus.cc:2229] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:29.674559 9818 raft_consensus.cc:2258] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:29.674587 9820 tablet_replica.cc:330] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: stopping tablet replica
I20260923 15:06:29.674685 9820 raft_consensus.cc:2229] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:29.674831 9820 raft_consensus.cc:2258] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:29.674960 9819 raft_consensus.cc:2258] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:29.675338 9818 ts_tablet_manager.cc:1905] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:29.675338 9820 ts_tablet_manager.cc:1905] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:29.675590 9416 tablet_service.cc:1467] Processing CreateTablet for tablet 1694df6f80274613aad23516d959d81b (DEFAULT_TABLE table=test-table [id=a9ba90a2239c403dbf3315f081e3697b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:29.675727 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1694df6f80274613aad23516d959d81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.676036 9819 ts_tablet_manager.cc:1905] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:29.677027 9797 tablet_bootstrap.cc:492] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Bootstrap starting.
I20260923 15:06:29.677086 9706 tablet_service.cc:1467] Processing CreateTablet for tablet 1694df6f80274613aad23516d959d81b (DEFAULT_TABLE table=test-table [id=a9ba90a2239c403dbf3315f081e3697b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:29.677186 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1694df6f80274613aad23516d959d81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.677460 9797 tablet_bootstrap.cc:654] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.678391 9795 tablet_bootstrap.cc:492] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Bootstrap starting.
I20260923 15:06:29.678670 9566 tablet_service.cc:1467] Processing CreateTablet for tablet 1694df6f80274613aad23516d959d81b (DEFAULT_TABLE table=test-table [id=a9ba90a2239c403dbf3315f081e3697b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:29.678704 9818 ts_tablet_manager.cc:1918] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:29.678805 9795 tablet_bootstrap.cc:654] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.678961 9566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1694df6f80274613aad23516d959d81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.679216 9818 log.cc:1196] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/f2eb6236ea7f407399f6ccd608d5107a
I20260923 15:06:29.679468 9797 tablet_bootstrap.cc:492] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: No bootstrap required, opened a new log
I20260923 15:06:29.679527 9797 ts_tablet_manager.cc:1397] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:06:29.679648 9818 ts_tablet_manager.cc:1939] T f2eb6236ea7f407399f6ccd608d5107a P a4cbff87688e409cb8a28d140cc885b0: Deleting consensus metadata
I20260923 15:06:29.679726 9797 raft_consensus.cc:348] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.679826 9797 raft_consensus.cc:374] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.679854 9797 raft_consensus.cc:729] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d06efda57324d60b34e6ff4eddbdd3b, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.679899 9797 consensus_queue.cc:260] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.679991 9797 ts_tablet_manager.cc:1428] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.680326 9796 tablet_bootstrap.cc:492] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Bootstrap starting.
I20260923 15:06:29.680809 9796 tablet_bootstrap.cc:654] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.680953 9819 ts_tablet_manager.cc:1918] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:29.681041 9819 log.cc:1196] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/f2eb6236ea7f407399f6ccd608d5107a
I20260923 15:06:29.681324 9819 ts_tablet_manager.cc:1939] T f2eb6236ea7f407399f6ccd608d5107a P a62deb54bc1547a1bceaf2630ba40bbc: Deleting consensus metadata
I20260923 15:06:29.681313 9274 catalog_manager.cc:4841] TS a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025): tablet f2eb6236ea7f407399f6ccd608d5107a (table test-table [id=37190efa0ba94d66a50f8b090ddffcf3]) successfully deleted
I20260923 15:06:29.681519 9820 ts_tablet_manager.cc:1918] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:29.681595 9820 log.cc:1196] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/f2eb6236ea7f407399f6ccd608d5107a
I20260923 15:06:29.681859 9820 ts_tablet_manager.cc:1939] T f2eb6236ea7f407399f6ccd608d5107a P 5d06efda57324d60b34e6ff4eddbdd3b: Deleting consensus metadata
I20260923 15:06:29.682283 9272 catalog_manager.cc:4841] TS 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65:44887): tablet f2eb6236ea7f407399f6ccd608d5107a (table test-table [id=37190efa0ba94d66a50f8b090ddffcf3]) successfully deleted
I20260923 15:06:29.682331 9274 catalog_manager.cc:4841] TS a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375): tablet f2eb6236ea7f407399f6ccd608d5107a (table test-table [id=37190efa0ba94d66a50f8b090ddffcf3]) successfully deleted
I20260923 15:06:29.682816 9796 tablet_bootstrap.cc:492] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: No bootstrap required, opened a new log
I20260923 15:06:29.682874 9796 ts_tablet_manager.cc:1397] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:06:29.683032 9796 raft_consensus.cc:348] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.683115 9796 raft_consensus.cc:374] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.683137 9796 raft_consensus.cc:729] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62deb54bc1547a1bceaf2630ba40bbc, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.683180 9796 consensus_queue.cc:260] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.683277 9796 ts_tablet_manager.cc:1428] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:29.689487 9795 tablet_bootstrap.cc:492] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: No bootstrap required, opened a new log
I20260923 15:06:29.689574 9795 ts_tablet_manager.cc:1397] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260923 15:06:29.689780 9795 raft_consensus.cc:348] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.689883 9795 raft_consensus.cc:374] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.689916 9795 raft_consensus.cc:729] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4cbff87688e409cb8a28d140cc885b0, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.689990 9795 consensus_queue.cc:260] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.690111 9795 ts_tablet_manager.cc:1428] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:29.708509 9806 raft_consensus.cc:482] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:29.708637 9806 raft_consensus.cc:504] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.708804 9806 leader_election.cc:288] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375), a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025)
I20260923 15:06:29.709064 9586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1694df6f80274613aad23516d959d81b" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" is_pre_election: true
I20260923 15:06:29.709112 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1694df6f80274613aad23516d959d81b" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4cbff87688e409cb8a28d140cc885b0" is_pre_election: true
I20260923 15:06:29.709196 9586 raft_consensus.cc:2454] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 0.
I20260923 15:06:29.709204 9726 raft_consensus.cc:2454] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 0.
I20260923 15:06:29.709347 9378 leader_election.cc:302] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d06efda57324d60b34e6ff4eddbdd3b, a4cbff87688e409cb8a28d140cc885b0; no voters:
I20260923 15:06:29.709465 9806 raft_consensus.cc:2781] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:29.709546 9806 raft_consensus.cc:482] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:29.709574 9806 raft_consensus.cc:3037] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.710511 9806 raft_consensus.cc:504] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.710732 9806 leader_election.cc:288] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 election: Requested vote from peers a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375), a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025)
I20260923 15:06:29.710904 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1694df6f80274613aad23516d959d81b" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4cbff87688e409cb8a28d140cc885b0"
I20260923 15:06:29.710901 9586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1694df6f80274613aad23516d959d81b" candidate_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62deb54bc1547a1bceaf2630ba40bbc"
I20260923 15:06:29.710984 9586 raft_consensus.cc:3037] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.710984 9726 raft_consensus.cc:3037] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.711627 9726 raft_consensus.cc:2454] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 1.
I20260923 15:06:29.711787 9586 raft_consensus.cc:2454] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d06efda57324d60b34e6ff4eddbdd3b in term 1.
I20260923 15:06:29.711796 9378 leader_election.cc:302] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d06efda57324d60b34e6ff4eddbdd3b, a4cbff87688e409cb8a28d140cc885b0; no voters:
I20260923 15:06:29.711874 9806 raft_consensus.cc:2781] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:29.711920 9806 raft_consensus.cc:686] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 LEADER]: Becoming Leader. State: Replica: 5d06efda57324d60b34e6ff4eddbdd3b, State: Running, Role: LEADER
I20260923 15:06:29.711974 9806 consensus_queue.cc:237] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.712662 9287 catalog_manager.cc:5495] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65). New cstate: current_term: 1 leader_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } health_report { overall_health: UNKNOWN } } }
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175988, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:29.752300 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.739214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.7.250.126:42681)
0923 15:06:29.739262 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.739264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.739363 (+ 99us) server_negotiation.cc:407] Connection header received
0923 15:06:29.739416 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.739417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.739429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.739490 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.739942 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.740698 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.741426 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.741735 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.741809 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.749185 (+ 7376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.749187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.749187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.749196 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.750978 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.751436 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.751437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.751437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.751504 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.751686 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.751687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.751687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.751839 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.751916 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.752096 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.752201 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:06:29.752444 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.739109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42681 (local address 127.0.0.1:39560)
0923 15:06:29.739281 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.739291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.739356 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.739510 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.739511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.739809 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:29.739859 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.739862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.740789 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.740790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.741331 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.741333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.741815 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.741816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.741872 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.747495 (+ 5623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:29.747501 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:29.749055 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:29.751091 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.751100 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.751102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.751320 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.751581 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:29.751581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:29.751582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:29.751640 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:29.751920 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:29.751928 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:29.751978 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.752045 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.752282 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":77,"thread_start_us":89,"threads_started":1}
I20260923 15:06:29.754921 9287 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:39560:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 15:06:29.755173 9287 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:29.756888 9566 tablet_service.cc:1467] Processing CreateTablet for tablet 5753a8046f124dfcaf0187353176fb8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=21eb4a0e99634381b7a38c852af333be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:29.756973 9706 tablet_service.cc:1467] Processing CreateTablet for tablet 5753a8046f124dfcaf0187353176fb8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=21eb4a0e99634381b7a38c852af333be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:29.757067 9566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5753a8046f124dfcaf0187353176fb8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.757126 9706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5753a8046f124dfcaf0187353176fb8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.758695 9796 tablet_bootstrap.cc:492] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: Bootstrap starting.
I20260923 15:06:29.758814 9416 tablet_service.cc:1467] Processing CreateTablet for tablet 5753a8046f124dfcaf0187353176fb8f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=21eb4a0e99634381b7a38c852af333be]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:29.758968 9416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5753a8046f124dfcaf0187353176fb8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:29.759140 9796 tablet_bootstrap.cc:654] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.759512 9795 tablet_bootstrap.cc:492] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: Bootstrap starting.
I20260923 15:06:29.759975 9795 tablet_bootstrap.cc:654] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.760483 9796 tablet_bootstrap.cc:492] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: No bootstrap required, opened a new log
I20260923 15:06:29.760540 9796 ts_tablet_manager.cc:1397] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:29.760723 9796 raft_consensus.cc:348] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.760839 9796 raft_consensus.cc:374] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.760871 9796 raft_consensus.cc:729] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62deb54bc1547a1bceaf2630ba40bbc, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.760919 9796 consensus_queue.cc:260] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.761036 9796 ts_tablet_manager.cc:1428] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:29.761135 9797 tablet_bootstrap.cc:492] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: Bootstrap starting.
I20260923 15:06:29.761107 9801 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.761250 9801 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.761560 9797 tablet_bootstrap.cc:654] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:29.762392 9797 tablet_bootstrap.cc:492] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: No bootstrap required, opened a new log
I20260923 15:06:29.762449 9797 ts_tablet_manager.cc:1397] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:29.762583 9797 raft_consensus.cc:348] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.762674 9797 raft_consensus.cc:374] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.762698 9797 raft_consensus.cc:729] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d06efda57324d60b34e6ff4eddbdd3b, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.762750 9797 consensus_queue.cc:260] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.762861 9797 ts_tablet_manager.cc:1428] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:29.762890 9802 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.762995 9802 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.763293 9795 tablet_bootstrap.cc:492] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: No bootstrap required, opened a new log
I20260923 15:06:29.763361 9795 ts_tablet_manager.cc:1397] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 15:06:29.763525 9795 raft_consensus.cc:348] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.763602 9795 raft_consensus.cc:374] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:29.763635 9795 raft_consensus.cc:729] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4cbff87688e409cb8a28d140cc885b0, State: Initialized, Role: FOLLOWER
I20260923 15:06:29.763685 9795 consensus_queue.cc:260] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.763774 9795 ts_tablet_manager.cc:1428] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:29.763926 9803 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.764025 9803 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.770691 9801 raft_consensus.cc:482] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:29.770792 9801 raft_consensus.cc:504] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.771163 9801 leader_election.cc:288] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65:44887), a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025)
I20260923 15:06:29.774905 9524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.771300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:44887 (local address 127.7.250.66:35683)
0923 15:06:29.771393 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.771431 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.771477 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.771756 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.771757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.771758 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.771806 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.771808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.772715 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.772716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.773808 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.773813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.774661 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.774662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.774736 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.774739 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.774788 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.774821 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":5}
I20260923 15:06:29.774916 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.771523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:56639 (local address 127.7.250.67:42025)
0923 15:06:29.771567 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.771569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.771579 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:29.771606 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.771607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.771616 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.771696 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.771911 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.772410 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.773935 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.774444 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.774491 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.774499 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.774788 (+ 289us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.774841 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:06:29.774966 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.771579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:35683 (local address 127.7.250.65:44887)
0923 15:06:29.771621 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.771623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.771630 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:29.771653 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.771654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.771662 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.771714 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:29.771891 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.772620 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.773915 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.774575 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.774665 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.774674 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.774853 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.774894 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:29.775281 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5753a8046f124dfcaf0187353176fb8f" candidate_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" is_pre_election: true
I20260923 15:06:29.775364 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.771326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42025 (local address 127.7.250.66:56639)
0923 15:06:29.771461 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.771471 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.771514 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.771714 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.771715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.771717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:29.771806 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.771808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.772502 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.772503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.773836 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.773838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.774568 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.774568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.774658 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.774662 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.774753 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.774815 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":2,"thread_start_us":80,"threads_started":1}
I20260923 15:06:29.775421 9441 raft_consensus.cc:2454] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a62deb54bc1547a1bceaf2630ba40bbc in term 0.
I20260923 15:06:29.775609 9516 leader_election.cc:302] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d06efda57324d60b34e6ff4eddbdd3b, a62deb54bc1547a1bceaf2630ba40bbc; no voters:
I20260923 15:06:29.775771 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5753a8046f124dfcaf0187353176fb8f" candidate_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4cbff87688e409cb8a28d140cc885b0" is_pre_election: true
I20260923 15:06:29.775887 9726 raft_consensus.cc:2454] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a62deb54bc1547a1bceaf2630ba40bbc in term 0.
I20260923 15:06:29.775974 9801 raft_consensus.cc:2781] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:29.776018 9801 raft_consensus.cc:482] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:29.776046 9801 raft_consensus.cc:3037] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.776800 9801 raft_consensus.cc:504] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.776948 9801 leader_election.cc:288] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [CANDIDATE]: Term 1 election: Requested vote from peers 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65:44887), a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025)
I20260923 15:06:29.777098 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5753a8046f124dfcaf0187353176fb8f" candidate_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4cbff87688e409cb8a28d140cc885b0"
I20260923 15:06:29.777170 9726 raft_consensus.cc:3037] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.777148 9441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5753a8046f124dfcaf0187353176fb8f" candidate_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d06efda57324d60b34e6ff4eddbdd3b"
I20260923 15:06:29.777230 9441 raft_consensus.cc:3037] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:29.777799 9726 raft_consensus.cc:2454] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a62deb54bc1547a1bceaf2630ba40bbc in term 1.
I20260923 15:06:29.778162 9441 raft_consensus.cc:2454] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a62deb54bc1547a1bceaf2630ba40bbc in term 1.
I20260923 15:06:29.778369 9516 leader_election.cc:302] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d06efda57324d60b34e6ff4eddbdd3b, a62deb54bc1547a1bceaf2630ba40bbc; no voters:
I20260923 15:06:29.778472 9801 raft_consensus.cc:2781] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:29.778640 9801 raft_consensus.cc:686] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [term 1 LEADER]: Becoming Leader. State: Replica: a62deb54bc1547a1bceaf2630ba40bbc, State: Running, Role: LEADER
I20260923 15:06:29.778712 9801 consensus_queue.cc:237] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } }
I20260923 15:06:29.779309 9835 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: TxnStatusTablet state changed. Reason: New leader a62deb54bc1547a1bceaf2630ba40bbc. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.779430 9835 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:29.779458 9287 catalog_manager.cc:5495] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc reported cstate change: term changed from 0 to 1, leader changed from <none> to a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66). New cstate: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:29.779682 9837 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 15:06:29.779834 9837 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 15:06:29.788838 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.786336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.7.250.66:41375)
0923 15:06:29.786552 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.786553 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.786559 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:06:29.786584 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.786584 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.786598 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.786671 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:29.786786 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.787275 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.788016 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.788227 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.788300 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.788484 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.788585 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.788668 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.788730 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.788763 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260923 15:06:29.788880 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.786330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41375 (local address 127.0.0.1:43064)
0923 15:06:29.786399 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.786411 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.786461 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.786679 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.786680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.786682 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:29.786733 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.786735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.787359 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.787361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.787929 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.787931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.788312 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.788313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.788372 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.788397 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.788654 (+ 257us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.788664 (+ 10us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.788724 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.788803 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 15:06:29.789979 9441 raft_consensus.cc:1261] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Refusing update from remote peer a62deb54bc1547a1bceaf2630ba40bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:29.790179 9726 raft_consensus.cc:1261] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Refusing update from remote peer a62deb54bc1547a1bceaf2630ba40bbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:29.790459 9835 consensus_queue.cc:1035] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:29.790563 9801 consensus_queue.cc:1035] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:29.791215 9803 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: TxnStatusTablet state changed. Reason: New leader a62deb54bc1547a1bceaf2630ba40bbc. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.791229 9802 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: TxnStatusTablet state changed. Reason: New leader a62deb54bc1547a1bceaf2630ba40bbc. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.791308 9803 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.791319 9802 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.794966 9801 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.795064 9801 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:29.795162 9802 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.795262 9802 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P 5d06efda57324d60b34e6ff4eddbdd3b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.795825 9835 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.795926 9835 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a62deb54bc1547a1bceaf2630ba40bbc: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:29.796360 9803 tablet_replica.cc:439] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d06efda57324d60b34e6ff4eddbdd3b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 44887 } } peers { permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 } } peers { permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 } } }
I20260923 15:06:29.796450 9803 tablet_replica.cc:442] T 5753a8046f124dfcaf0187353176fb8f P a4cbff87688e409cb8a28d140cc885b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:29.801513 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.799077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:33275 (local address 127.7.250.66:41375)
0923 15:06:29.799122 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.799124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.799262 (+ 138us) server_negotiation.cc:407] Connection header received
0923 15:06:29.799326 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.799327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.799338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.799387 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:29.799534 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.800006 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.800785 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.800994 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.801066 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.801261 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.801295 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.801364 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.801426 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.801446 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:06:29.801554 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.799037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41375 (local address 127.7.250.65:33275)
0923 15:06:29.799184 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.799194 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.799275 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.799388 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.799389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.799390 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:29.799462 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.799463 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.800093 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.800094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.800686 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.800688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.801078 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.801079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.801151 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.801165 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.801367 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.801369 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.801421 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.801468 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":73,"threads_started":1}
I20260923 15:06:29.803547 9726 raft_consensus.cc:1261] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Refusing update from remote peer 5d06efda57324d60b34e6ff4eddbdd3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:29.803612 9586 raft_consensus.cc:1261] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Refusing update from remote peer 5d06efda57324d60b34e6ff4eddbdd3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:29.803793 9806 consensus_queue.cc:1035] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4cbff87688e409cb8a28d140cc885b0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:29.803895 9802 consensus_queue.cc:1035] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a62deb54bc1547a1bceaf2630ba40bbc" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:29.812003 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.809326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:36667 (local address 127.7.250.65:44887)
0923 15:06:29.809363 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.809366 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.809520 (+ 154us) server_negotiation.cc:407] Connection header received
0923 15:06:29.809583 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.809584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.809594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.809648 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:29.809780 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.810244 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.811283 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.811489 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.811575 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.811752 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.811787 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.811855 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.811915 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.811934 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:06:29.812045 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.809106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:44887 (local address 127.7.250.66:36667)
0923 15:06:29.809442 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:29.809454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:29.809535 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:29.809651 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:29.809652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:29.809653 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:29.809716 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.809717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.810349 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.810350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.811189 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:29.811192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.811576 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.811577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:29.811643 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.811664 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:29.811856 (+ 192us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:29.811858 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:29.811911 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 15:06:29.811956 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":97,"threads_started":1}
I20260923 15:06:29.820391 9287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:29.820495 9287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:29.821147 9287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdaf64a820474395ad9d4ca0e75f8de4: Sending DeleteTablet for 3 replicas of tablet 1694df6f80274613aad23516d959d81b
I20260923 15:06:29.821416 9416 tablet_service.cc:1514] Processing DeleteTablet for tablet 1694df6f80274613aad23516d959d81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:29 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:52126
I20260923 15:06:29.821532 9820 tablet_replica.cc:330] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: stopping tablet replica
I20260923 15:06:29.821597 9820 raft_consensus.cc:2229] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:29.821650 9820 raft_consensus.cc:2258] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:29.821883 9706 tablet_service.cc:1514] Processing DeleteTablet for tablet 1694df6f80274613aad23516d959d81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:29 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:55164
I20260923 15:06:29.821998 9818 tablet_replica.cc:330] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: stopping tablet replica
I20260923 15:06:29.822033 9820 ts_tablet_manager.cc:1905] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:29.822057 9818 raft_consensus.cc:2229] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:29.822089 9818 raft_consensus.cc:2258] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:29.823874 9566 tablet_service.cc:1514] Processing DeleteTablet for tablet 1694df6f80274613aad23516d959d81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:29 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:43054
I20260923 15:06:29.823966 9819 tablet_replica.cc:330] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: stopping tablet replica
I20260923 15:06:29.823954 9820 ts_tablet_manager.cc:1918] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:29.824007 9819 raft_consensus.cc:2229] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:29.824011 9820 log.cc:1196] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/1694df6f80274613aad23516d959d81b
I20260923 15:06:29.824036 9819 raft_consensus.cc:2258] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:29.824241 9820 ts_tablet_manager.cc:1939] T 1694df6f80274613aad23516d959d81b P 5d06efda57324d60b34e6ff4eddbdd3b: Deleting consensus metadata
I20260923 15:06:29.824338 9819 ts_tablet_manager.cc:1905] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:29.825596 9272 catalog_manager.cc:4841] TS 5d06efda57324d60b34e6ff4eddbdd3b (127.7.250.65:44887): tablet 1694df6f80274613aad23516d959d81b (table test-table [id=a9ba90a2239c403dbf3315f081e3697b]) successfully deleted
I20260923 15:06:29.825816 9819 ts_tablet_manager.cc:1918] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:29.825865 9819 log.cc:1196] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/1694df6f80274613aad23516d959d81b
I20260923 15:06:29.825886 9818 ts_tablet_manager.cc:1905] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:29.826097 9819 ts_tablet_manager.cc:1939] T 1694df6f80274613aad23516d959d81b P a62deb54bc1547a1bceaf2630ba40bbc: Deleting consensus metadata
I20260923 15:06:29.826454 9274 catalog_manager.cc:4841] TS a62deb54bc1547a1bceaf2630ba40bbc (127.7.250.66:41375): tablet 1694df6f80274613aad23516d959d81b (table test-table [id=a9ba90a2239c403dbf3315f081e3697b]) successfully deleted
I20260923 15:06:29.827416 9818 ts_tablet_manager.cc:1918] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:29.827484 9818 log.cc:1196] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/1694df6f80274613aad23516d959d81b
I20260923 15:06:29.827685 9818 ts_tablet_manager.cc:1939] T 1694df6f80274613aad23516d959d81b P a4cbff87688e409cb8a28d140cc885b0: Deleting consensus metadata
I20260923 15:06:29.828442 9274 catalog_manager.cc:4841] TS a4cbff87688e409cb8a28d140cc885b0 (127.7.250.67:42025): tablet 1694df6f80274613aad23516d959d81b (table test-table [id=a9ba90a2239c403dbf3315f081e3697b]) successfully deleted
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9243](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175989, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:29.834475 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.822438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.7.250.65:44887)
0923 15:06:29.822507 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.822510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.822740 (+ 230us) server_negotiation.cc:407] Connection header received
0923 15:06:29.822841 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.822842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.822873 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.822931 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.823623 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.824488 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.825568 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.825873 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.825973 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.831118 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.831135 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.831136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.831144 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.832919 (+ 1775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.833383 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.833387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.833387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.833464 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.833786 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.833787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.833787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.833962 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.834158 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.834216 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.834344 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260923 15:06:29.834774 9468 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52140
I20260923 15:06:29.843096 9337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:29.835787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.7.250.126:42681)
0923 15:06:29.835842 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:29.835844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:29.836012 (+ 168us) server_negotiation.cc:407] Connection header received
0923 15:06:29.836046 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:29.836047 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:29.836059 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:29.836132 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:29.836710 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.837496 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.838172 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:29.838495 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:29.838568 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:29.840220 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:29.840221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:29.840222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:29.840229 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:29.841888 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.842222 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.842223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.842223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.842381 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:29.842571 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:29.842571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:29.842572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:29.842711 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:29.842817 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:29.842919 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 15:06:29.842997 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260923 15:06:29.843350 9287 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39570
I20260923 15:06:29.843989 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9360
I20260923 15:06:29.852926 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9500
I20260923 15:06:29.858964 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9641
I20260923 15:06:29.863915 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9256
2026-09-23T15:06:29Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2245 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9858](info): set up 2 sockets
Sep 23 15:06:29 dist-test-slave-bb3x krb5kdc[9858](info): commencing operation
krb5kdc: starting...
W20260923 15:06:30.931555 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:31.057507 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45265
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:45265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:31.140412 9871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:31.140615 9871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:31.140638 9871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:31.141801 9871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:31.141839 9871 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 15:06:31.141852 9871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:31.141866 9871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:31.141878 9871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:31.141891 9871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:31.143358 9871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:45265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:31.143549 9871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:31.143724 9871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:31.148617 9876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.148689 9879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.148612 9877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:31.148829 9871 server_base.cc:1034] running on GCE node
I20260923 15:06:31.149085 9871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:31.149385 9871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:31.150606 9871 hybrid_clock.cc:648] HybridClock initialized: now 1790175991150592 us; error 28 us; skew 500 ppm
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:31.153155 9871 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:31.153746 9871 webserver.cc:460] Webserver started at http://127.7.250.126:42723/ using document root <none> and password file <none>
I20260923 15:06:31.153992 9871 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:31.154060 9871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:31.154215 9871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:31.155341 9871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "26ac7307224d4e78b2765cc6bd5b807d"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "03e6d06056be27667847cc44aa835f59"
server_key_iv: "2ca28733e5cd0ce201db076a276dbf65"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.155709 9871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "26ac7307224d4e78b2765cc6bd5b807d"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "03e6d06056be27667847cc44aa835f59"
server_key_iv: "2ca28733e5cd0ce201db076a276dbf65"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.157088 9871 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:31.157872 9886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.158051 9871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:31.158152 9871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "26ac7307224d4e78b2765cc6bd5b807d"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "03e6d06056be27667847cc44aa835f59"
server_key_iv: "2ca28733e5cd0ce201db076a276dbf65"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.158241 9871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:31.177491 9871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:31.179702 9871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:31.179899 9871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:31.185070 9871 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:45265
I20260923 15:06:31.185187 9948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:45265 every 8 connection(s)
I20260923 15:06:31.185500 9871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:31.186335 9949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.188650 9949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Bootstrap starting.
I20260923 15:06:31.189257 9949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.189549 9949 log.cc:824] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:31.190316 9949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: No bootstrap required, opened a new log
I20260923 15:06:31.192499 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9871
I20260923 15:06:31.192600 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:31.192519 9949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ac7307224d4e78b2765cc6bd5b807d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45265 } }
I20260923 15:06:31.192704 9949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.192749 9949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ac7307224d4e78b2765cc6bd5b807d, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.192782 8169 external_mini_cluster.cc:1428] Setting key 29ccfa4a7c940d4c526de66e80a97573
I20260923 15:06:31.192902 9949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ac7307224d4e78b2765cc6bd5b807d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45265 } }
I20260923 15:06:31.192993 9949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:31.193017 9949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:31.193089 9949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:31.193851 9949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ac7307224d4e78b2765cc6bd5b807d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45265 } }
I20260923 15:06:31.193960 9949 leader_election.cc:302] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26ac7307224d4e78b2765cc6bd5b807d; no voters:
I20260923 15:06:31.194124 9949 leader_election.cc:288] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:31.194154 9954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:31.194339 9954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [term 1 LEADER]: Becoming Leader. State: Replica: 26ac7307224d4e78b2765cc6bd5b807d, State: Running, Role: LEADER
I20260923 15:06:31.194425 9949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:31.194437 9954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ac7307224d4e78b2765cc6bd5b807d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45265 } }
I20260923 15:06:31.194795 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26ac7307224d4e78b2765cc6bd5b807d. Latest consensus state: current_term: 1 leader_uuid: "26ac7307224d4e78b2765cc6bd5b807d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ac7307224d4e78b2765cc6bd5b807d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45265 } } }
I20260923 15:06:31.194859 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:31.194983 9956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26ac7307224d4e78b2765cc6bd5b807d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ac7307224d4e78b2765cc6bd5b807d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45265 } } }
I20260923 15:06:31.195078 9956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:31.195948 9959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:31.196518 9959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:31.197942 9959 catalog_manager.cc:1347] Generated new cluster ID: 6276a8cef18a4adfa518b806793b4c03
I20260923 15:06:31.197992 9959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:31.204947 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.193435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40740 (local address 127.7.250.126:45265)
0923 15:06:31.193639 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.193649 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.193691 (+ 42us) server_negotiation.cc:407] Connection header received
0923 15:06:31.193807 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.193815 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.193875 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.194863 (+ 988us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.194972 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.196445 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.197556 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.198162 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.198299 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.201195 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.201200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.201200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.201213 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.203232 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.203682 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.203685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.203686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.203802 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.204017 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.204017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.204018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.204330 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.204477 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.204617 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.204749 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":80,"threads_started":1}
I20260923 15:06:31.211486 9959 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:31.212034 9959 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:31.219008 9959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Generated new TSK 0
I20260923 15:06:31.219216 9959 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:31.250380 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45265
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:31.332739 9975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:31.332976 9975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:31.333019 9975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:31.334160 9975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:31.334225 9975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:31.334259 9975 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 15:06:31.334334 9975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:31.334365 9975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:31.335817 9975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:45265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:31.336084 9975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:31.336318 9975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:31.341370 9981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.341355 9983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.341351 9980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:31.341642 9975 server_base.cc:1034] running on GCE node
I20260923 15:06:31.341827 9975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:31.342012 9975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:31.343225 9975 hybrid_clock.cc:648] HybridClock initialized: now 1790175991343210 us; error 28 us; skew 500 ppm
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:31.345822 9975 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:31.346315 9975 webserver.cc:460] Webserver started at http://127.7.250.65:38121/ using document root <none> and password file <none>
I20260923 15:06:31.346554 9975 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:31.346624 9975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:31.346768 9975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:31.347774 9975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "cbf2942fb965445db54e6f1985e89499"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "11dc8ffccdaa823ababe048c8954a74c"
server_key_iv: "b1dab9635adc5616244ad5e656d061c0"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.348133 9975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "cbf2942fb965445db54e6f1985e89499"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "11dc8ffccdaa823ababe048c8954a74c"
server_key_iv: "b1dab9635adc5616244ad5e656d061c0"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.349525 9975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:31.350230 9990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.350433 9975 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:31.350529 9975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "cbf2942fb965445db54e6f1985e89499"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "11dc8ffccdaa823ababe048c8954a74c"
server_key_iv: "b1dab9635adc5616244ad5e656d061c0"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.350612 9975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:31.369438 9975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:31.371912 9975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:31.372090 9975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:31.372623 9975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:31.372684 9975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.372730 9975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:31.372776 9975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:31.380235 9975 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:46467
I20260923 15:06:31.380316 10105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:46467 every 8 connection(s)
I20260923 15:06:31.380654 9975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:31.383036 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.373221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:39959 (local address 127.7.250.126:45265)
0923 15:06:31.373270 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.373272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.373511 (+ 239us) server_negotiation.cc:407] Connection header received
0923 15:06:31.374048 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.374049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.374065 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.374137 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.375187 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.376094 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.377136 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.377389 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.377485 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.380019 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.380020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.380021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.380029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.381597 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.382006 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.382007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.382008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.382066 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.382371 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.382372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.382372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.382499 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.382577 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.382859 (+ 282us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.382946 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:31.383222 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.373211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.7.250.65:39959)
0923 15:06:31.373390 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.373424 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.373937 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.374285 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.374289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.374713 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.375034 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.375040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.376213 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.376214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.377036 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.377039 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.377512 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.377514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.377671 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.378226 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.378234 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.379915 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.381692 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.381700 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.381702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.381926 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.382143 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.382144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.382145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.382314 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.382568 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.382575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.382720 (+ 145us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.382826 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.382985 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260923 15:06:31.383744 10106 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45265
I20260923 15:06:31.383834 10106 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:31.384030 10106 heartbeater.cc:511] Master 127.7.250.126:45265 requested a full tablet report, sending...
I20260923 15:06:31.384603 9903 ts_manager.cc:194] Registered new tserver with Master: cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467)
I20260923 15:06:31.385309 9903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:39959
I20260923 15:06:31.385666 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 9975
I20260923 15:06:31.385762 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:31.385936 8169 external_mini_cluster.cc:1428] Setting key 3bf6a5d6e780a81090942ea6a37e8d66
I20260923 15:06:31.391153 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.384763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:49199 (local address 127.7.250.126:45265)
0923 15:06:31.384808 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.384810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.385020 (+ 210us) server_negotiation.cc:407] Connection header received
0923 15:06:31.385092 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.385093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.385104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.385168 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.385507 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.386240 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.387237 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.387420 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.387482 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.388531 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.388533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.388533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.388541 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.389989 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.390433 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.390434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.390434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.390485 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.390694 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.390694 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.390695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.390796 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.390847 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.391012 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.391074 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:31.391263 10112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.384808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.7.250.65:49199)
0923 15:06:31.384951 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.384962 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.385035 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.385154 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.385155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.385335 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.385421 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.385423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.386365 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.386366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.387140 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.387142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.387498 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.387499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.387583 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.388143 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.388154 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.388442 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.390079 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.390080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.390081 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.390350 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.390554 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.390555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.390556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.390627 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.390863 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.390864 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.390915 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.390997 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.391120 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:31.414204 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45265
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:31.494287 10115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:31.494556 10115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:31.494598 10115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:31.495869 10115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:31.495934 10115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:31.495968 10115 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 15:06:31.496016 10115 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:31.496054 10115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:31.497574 10115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:45265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:31.497859 10115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:31.498103 10115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:31.503158 10123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.503182 10120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.503182 10121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:31.503520 10115 server_base.cc:1034] running on GCE node
I20260923 15:06:31.503724 10115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:31.503963 10115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:31.505134 10115 hybrid_clock.cc:648] HybridClock initialized: now 1790175991505109 us; error 37 us; skew 500 ppm
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:31.507663 10115 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:31.508114 10115 webserver.cc:460] Webserver started at http://127.7.250.66:37879/ using document root <none> and password file <none>
I20260923 15:06:31.508338 10115 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:31.508383 10115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:31.508539 10115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:31.509582 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "5b04974649ad477ea37705e80acd3d1f"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "99e06416a67bac2be660261c40f3d8b8"
server_key_iv: "7fa2574f000a97fd2c60a77bf61451a9"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.509883 10115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "5b04974649ad477ea37705e80acd3d1f"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "99e06416a67bac2be660261c40f3d8b8"
server_key_iv: "7fa2574f000a97fd2c60a77bf61451a9"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.511257 10115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:31.511958 10130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.512159 10115 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:31.512269 10115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "5b04974649ad477ea37705e80acd3d1f"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "99e06416a67bac2be660261c40f3d8b8"
server_key_iv: "7fa2574f000a97fd2c60a77bf61451a9"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.512351 10115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:31.528306 10115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:31.530416 10115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:31.530563 10115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:31.531184 10115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:31.531234 10115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.531270 10115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:31.531288 10115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:31.538686 10115 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:37967
I20260923 15:06:31.538780 10245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:37967 every 8 connection(s)
I20260923 15:06:31.539057 10115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:06:31.541126 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.531686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:40117 (local address 127.7.250.126:45265)
0923 15:06:31.531760 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.531763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.531999 (+ 236us) server_negotiation.cc:407] Connection header received
0923 15:06:31.532522 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.532523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.532536 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.532592 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.533522 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.534067 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.535194 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.535398 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.535481 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.538089 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.538090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.538090 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.538099 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.539737 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.540153 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.540154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.540155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.540215 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.540468 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.540469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.540470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.540591 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.540673 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.540954 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.541034 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:06:31.541307 10139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.531693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.7.250.66:40117)
0923 15:06:31.531862 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.531903 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.532418 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.532731 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.532736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.533119 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.533415 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.533423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.534174 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.534175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.535090 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.535093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.535510 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.535512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.535652 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.536239 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.536250 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.537992 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.539833 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.539839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.539841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.540056 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.540286 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.540287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.540288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.540400 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.540665 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.540672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.540811 (+ 139us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.540915 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.541073 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260923 15:06:31.541805 10246 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45265
I20260923 15:06:31.541889 10246 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:31.542078 10246 heartbeater.cc:511] Master 127.7.250.126:45265 requested a full tablet report, sending...
I20260923 15:06:31.542513 9903 ts_manager.cc:194] Registered new tserver with Master: 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:31.543031 9903 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:40117
I20260923 15:06:31.548259 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.542887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:50521 (local address 127.7.250.126:45265)
0923 15:06:31.543169 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.543171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.543179 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:31.543197 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.543198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.543213 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.543282 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.543646 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.544102 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.544791 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.544966 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.545016 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.545961 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.545962 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.545962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.545967 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.547281 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.547639 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.547639 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.547639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.547685 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.547853 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.547854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.547854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.547980 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.548030 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.548136 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.548203 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260923 15:06:31.548272 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.542816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.7.250.66:50521)
0923 15:06:31.542948 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.542959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.543012 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.543289 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.543291 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.543490 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.543566 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.543568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.544188 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.544189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.544687 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.544688 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.545028 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.545029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.545155 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.545615 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.545620 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.545893 (+ 273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.547353 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.547354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.547354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.547518 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.547741 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.547742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.547742 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.547795 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.548034 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.548034 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.548071 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.548121 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.548194 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
I20260923 15:06:31.549080 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 10115
I20260923 15:06:31.549183 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:31.549386 8169 external_mini_cluster.cc:1428] Setting key b3ca4e3c8c518601cc4a0c366ad9f292
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:31.575806 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45265
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:31.658540 10255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:31.658802 10255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:31.658829 10255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:31.659956 10255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:31.659992 10255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:31.660012 10255 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 15:06:31.660024 10255 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:31.660040 10255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:31.661736 10255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:45265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:31.661957 10255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:31.662133 10255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:31.667141 10261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:31.667212 10255 server_base.cc:1034] running on GCE node
W20260923 15:06:31.667141 10263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:31.667157 10260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:31.667557 10255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:31.667837 10255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:31.668987 10255 hybrid_clock.cc:648] HybridClock initialized: now 1790175991668972 us; error 28 us; skew 500 ppm
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:31.671684 10255 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:31.672079 10255 webserver.cc:460] Webserver started at http://127.7.250.67:38513/ using document root <none> and password file <none>
I20260923 15:06:31.672320 10255 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:31.672365 10255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:31.672458 10255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:31.673604 10255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "fb16d0bc51814f47bbf0d1f94f323002"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "13012d89b38ac1af469aa1a581e0717e"
server_key_iv: "371dcc7294cfb3045b9287be14df5031"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.674255 10255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "fb16d0bc51814f47bbf0d1f94f323002"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "13012d89b38ac1af469aa1a581e0717e"
server_key_iv: "371dcc7294cfb3045b9287be14df5031"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.675740 10255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:31.676621 10270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.676888 10255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:31.677012 10255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "fb16d0bc51814f47bbf0d1f94f323002"
format_stamp: "Formatted at 2026-09-23 15:06:31 on dist-test-slave-bb3x"
server_key: "13012d89b38ac1af469aa1a581e0717e"
server_key_iv: "371dcc7294cfb3045b9287be14df5031"
server_key_version: "encryptionkey@0"
I20260923 15:06:31.677134 10255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:31.690464 10255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:31.693082 10255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:31.693264 10255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:31.693821 10255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:31.693930 10255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.694012 10255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:31.694048 10255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:31.702615 10255 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:42811
I20260923 15:06:31.702674 10385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:42811 every 8 connection(s)
I20260923 15:06:31.703073 10255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:31.705592 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.694398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:51013 (local address 127.7.250.126:45265)
0923 15:06:31.694482 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.694485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.694840 (+ 355us) server_negotiation.cc:407] Connection header received
0923 15:06:31.695678 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.695684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.695695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.695741 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.696797 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.697647 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.699561 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.699776 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.699854 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.702049 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.702051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.702051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.702059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.703882 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.704442 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.704443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.704444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.704507 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.704861 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.704861 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.704862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.705004 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.705101 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.705412 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.705501 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:06:31.705780 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.694409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.7.250.67:51013)
0923 15:06:31.694676 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.694730 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.695546 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.695848 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.695853 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.696310 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.696681 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.696689 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.697765 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.697766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.699443 (+ 1677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.699446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.699875 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.699876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.700017 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.700544 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.700552 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.701950 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.703983 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.703989 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.703991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.704309 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.704679 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.704680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.704680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.704785 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.705094 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.705101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.705263 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.705375 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.705541 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":129,"threads_started":1}
I20260923 15:06:31.706367 10386 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45265
I20260923 15:06:31.706460 10386 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:31.706733 10386 heartbeater.cc:511] Master 127.7.250.126:45265 requested a full tablet report, sending...
I20260923 15:06:31.707176 9902 ts_manager.cc:194] Registered new tserver with Master: fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811)
I20260923 15:06:31.707672 9902 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:51013
I20260923 15:06:31.711431 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 10255
I20260923 15:06:31.711530 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:31.711714 8169 external_mini_cluster.cc:1428] Setting key 392b07a399a0eb856cb08b8fabca5b54
I20260923 15:06:31.712882 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:06:31.714306 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.707442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:53135 (local address 127.7.250.126:45265)
0923 15:06:31.707484 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.707486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.707968 (+ 482us) server_negotiation.cc:407] Connection header received
0923 15:06:31.708014 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.708015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.708025 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.708105 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.708484 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.709036 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.709851 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.710072 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.710157 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.711156 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.711157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.711158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.711174 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.713039 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.713451 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.713452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.713453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.713513 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.713783 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.713784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.713784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.713900 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.713963 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.714130 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.714193 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:31.714416 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.707381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.7.250.67:53135)
0923 15:06:31.707860 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.707887 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.707958 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.708131 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.708132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.708320 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.708388 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.708390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.709141 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.709143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.709739 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.709742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.710160 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.710161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.710246 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.710729 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.710734 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.711060 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.713139 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.713140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.713140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.713355 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.713602 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.713603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.713603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.713689 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.713975 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.713976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.714031 (+ 55us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.714107 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.714233 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":71,"threads_started":1}
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:31.736474 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.729444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40748 (local address 127.7.250.126:45265)
0923 15:06:31.729509 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.729510 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.729519 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:31.729545 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.729546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.729555 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.729612 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.730198 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.730801 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.731507 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.731712 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.731763 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.733860 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.733861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.733861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.733868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.735315 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.735720 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.735721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.735721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.735772 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.736030 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.736030 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.736031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.736138 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.736234 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.736347 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.736408 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:06:31.738458 9902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:31.740036 9902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:31.747288 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.742832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:37967 (local address 127.0.0.1:34236)
0923 15:06:31.743623 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.743633 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.743679 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.743849 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.743850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.743851 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:31.743904 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.743905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.744863 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.744864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.745865 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.745868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.746950 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.746951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.747030 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.747044 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.747121 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.747183 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"mutex_wait_us":20,"thread_start_us":72,"threads_started":1}
I20260923 15:06:31.747299 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.743074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.7.250.66:37967)
0923 15:06:31.743264 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.743267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.743681 (+ 414us) server_negotiation.cc:407] Connection header received
0923 15:06:31.743748 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.743750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.743762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.743821 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:31.743969 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.744784 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.745968 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.746853 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.746942 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.747011 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.747155 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.747196 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260923 15:06:31.747292 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.742598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.0.0.1:37044)
0923 15:06:31.742866 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.742913 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.742987 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.743359 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.743362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.743364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:31.743463 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.743464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.744589 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.744590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.745944 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.745947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.746991 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.746992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.747079 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.747083 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.747142 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.747189 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":1,"thread_start_us":66,"threads_started":1}
I20260923 15:06:31.747869 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.742938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.7.250.67:42811)
0923 15:06:31.743180 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.743185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.743197 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:31.743240 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.743243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.743260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.743352 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:31.743543 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.744500 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.746016 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.746852 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.747607 (+ 755us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.747689 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.747723 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.747770 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":97,"threads_started":1}
I20260923 15:06:31.748298 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.742667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:46467 (local address 127.0.0.1:32996)
0923 15:06:31.743453 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.743463 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.743539 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.744746 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.744747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.744749 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:31.744835 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.744837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.746330 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.746331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.747133 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.747135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.748102 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.748103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.748162 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.748166 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.748196 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.748233 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":28,"thread_start_us":92,"threads_started":1}
I20260923 15:06:31.748363 10180 tablet_service.cc:1467] Processing CreateTablet for tablet 412a74d54d6c4ba1b4998941fa4ee692 (DEFAULT_TABLE table=test-table [id=30a1d4c4b4a341d3a86d37034c72ce47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:31.748627 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.742936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.7.250.65:46467)
0923 15:06:31.743190 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.743195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.744585 (+ 1390us) server_negotiation.cc:407] Connection header received
0923 15:06:31.744628 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.744630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.744643 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.744733 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:31.744904 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.746181 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.747219 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.747991 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.748406 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.748468 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.748494 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.748533 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":93,"threads_started":1}
I20260923 15:06:31.748601 10320 tablet_service.cc:1467] Processing CreateTablet for tablet 412a74d54d6c4ba1b4998941fa4ee692 (DEFAULT_TABLE table=test-table [id=30a1d4c4b4a341d3a86d37034c72ce47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:31.748775 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412a74d54d6c4ba1b4998941fa4ee692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.748960 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412a74d54d6c4ba1b4998941fa4ee692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.749221 10037 tablet_service.cc:1467] Processing CreateTablet for tablet 412a74d54d6c4ba1b4998941fa4ee692 (DEFAULT_TABLE table=test-table [id=30a1d4c4b4a341d3a86d37034c72ce47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:31.749527 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 412a74d54d6c4ba1b4998941fa4ee692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.752028 10409 tablet_bootstrap.cc:492] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Bootstrap starting.
I20260923 15:06:31.752665 10410 tablet_bootstrap.cc:492] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Bootstrap starting.
I20260923 15:06:31.753038 10411 tablet_bootstrap.cc:492] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Bootstrap starting.
I20260923 15:06:31.753369 10410 tablet_bootstrap.cc:654] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.753369 10409 tablet_bootstrap.cc:654] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.753634 10411 tablet_bootstrap.cc:654] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.753787 10410 log.cc:824] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:31.753844 10409 log.cc:824] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:31.754045 10411 log.cc:824] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:31.754823 10409 tablet_bootstrap.cc:492] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: No bootstrap required, opened a new log
I20260923 15:06:31.754899 10411 tablet_bootstrap.cc:492] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: No bootstrap required, opened a new log
I20260923 15:06:31.754911 10409 ts_tablet_manager.cc:1397] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:06:31.754976 10411 ts_tablet_manager.cc:1397] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:31.756255 10409 raft_consensus.cc:348] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.756409 10409 raft_consensus.cc:374] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.756454 10409 raft_consensus.cc:729] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.756464 10411 raft_consensus.cc:348] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.756578 10409 consensus_queue.cc:260] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.756635 10411 raft_consensus.cc:374] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.756685 10411 raft_consensus.cc:729] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.756834 10409 ts_tablet_manager.cc:1428] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:31.756807 10411 consensus_queue.cc:260] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.756916 10386 heartbeater.cc:503] Master 127.7.250.126:45265 was elected leader, sending a full tablet report...
I20260923 15:06:31.757128 10411 ts_tablet_manager.cc:1428] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:31.757131 10106 heartbeater.cc:503] Master 127.7.250.126:45265 was elected leader, sending a full tablet report...
I20260923 15:06:31.757233 10410 tablet_bootstrap.cc:492] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: No bootstrap required, opened a new log
I20260923 15:06:31.757313 10410 ts_tablet_manager.cc:1397] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 15:06:31.759038 10410 raft_consensus.cc:348] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.759194 10410 raft_consensus.cc:374] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.759240 10410 raft_consensus.cc:729] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b04974649ad477ea37705e80acd3d1f, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.759351 10410 consensus_queue.cc:260] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.759624 10410 ts_tablet_manager.cc:1428] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:06:31.759662 10246 heartbeater.cc:503] Master 127.7.250.126:45265 was elected leader, sending a full tablet report...
I20260923 15:06:31.774542 10416 raft_consensus.cc:482] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:31.774693 10416 raft_consensus.cc:504] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.775156 10416 leader_election.cc:288] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811), 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:31.778568 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.775353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.7.250.65:51173)
0923 15:06:31.775418 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.775429 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.775479 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.775741 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.775742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.775745 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:31.775821 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.775824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.776497 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.776498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.777324 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.777326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.778206 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.778207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.778344 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.778349 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.778405 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.778478 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 15:06:31.778633 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.775445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:51173 (local address 127.7.250.67:42811)
0923 15:06:31.775531 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.775533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.775544 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:31.775575 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.775576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.775586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.775714 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:31.775900 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.776424 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.777423 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.778132 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.778210 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.778221 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.778520 (+ 299us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.778553 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 15:06:31.778964 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412a74d54d6c4ba1b4998941fa4ee692" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb16d0bc51814f47bbf0d1f94f323002" is_pre_election: true
I20260923 15:06:31.779040 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.775472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:37967 (local address 127.7.250.65:41819)
0923 15:06:31.775650 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.775661 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.775737 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.775899 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.775900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.775902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:31.775967 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.775969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.776735 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.776736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.777741 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.777744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.778485 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.778486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.778573 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.778577 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.778938 (+ 361us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.778972 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260923 15:06:31.779078 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.775599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:41819 (local address 127.7.250.66:37967)
0923 15:06:31.775641 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.775643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.775730 (+ 87us) server_negotiation.cc:407] Connection header received
0923 15:06:31.775797 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.775798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.775808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.775870 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:31.776040 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.776639 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.777837 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.778404 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.778475 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.778481 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.778968 (+ 487us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.779015 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:31.779129 10340 raft_consensus.cc:2454] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 0.
I20260923 15:06:31.779342 9991 leader_election.cc:302] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbf2942fb965445db54e6f1985e89499, fb16d0bc51814f47bbf0d1f94f323002; no voters:
I20260923 15:06:31.779353 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412a74d54d6c4ba1b4998941fa4ee692" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f" is_pre_election: true
I20260923 15:06:31.779484 10416 raft_consensus.cc:2781] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:31.779541 10416 raft_consensus.cc:482] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:31.779572 10416 raft_consensus.cc:3037] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:31.779496 10200 raft_consensus.cc:2454] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 0.
I20260923 15:06:31.780438 10416 raft_consensus.cc:504] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.780576 10416 leader_election.cc:288] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 election: Requested vote from peers fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811), 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:31.780727 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412a74d54d6c4ba1b4998941fa4ee692" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f"
I20260923 15:06:31.780838 10200 raft_consensus.cc:3037] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:31.780825 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "412a74d54d6c4ba1b4998941fa4ee692" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb16d0bc51814f47bbf0d1f94f323002"
I20260923 15:06:31.780905 10340 raft_consensus.cc:3037] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:31.781716 10200 raft_consensus.cc:2454] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 1.
I20260923 15:06:31.781716 10340 raft_consensus.cc:2454] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 1.
I20260923 15:06:31.781886 9991 leader_election.cc:302] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, cbf2942fb965445db54e6f1985e89499; no voters:
I20260923 15:06:31.781986 10416 raft_consensus.cc:2781] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:31.782135 10416 raft_consensus.cc:686] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 1 LEADER]: Becoming Leader. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Running, Role: LEADER
I20260923 15:06:31.782236 10416 consensus_queue.cc:237] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } }
I20260923 15:06:31.783133 9903 catalog_manager.cc:5495] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbf2942fb965445db54e6f1985e89499 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "cbf2942fb965445db54e6f1985e89499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } health_report { overall_health: UNKNOWN } } }
W20260923 15:06:31.789686 10247 tablet.cc:2369] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 15:06:31 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175991, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:31.810381 9953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.803163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40754 (local address 127.7.250.126:45265)
0923 15:06:31.803255 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:31.803258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:31.803344 (+ 86us) server_negotiation.cc:407] Connection header received
0923 15:06:31.803393 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:31.803394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:31.803404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:31.803461 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:31.803968 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.804472 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.805219 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.805398 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.805455 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.807742 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:31.807743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:31.807744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:31.807750 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:31.809145 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.809511 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.809512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.809513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.809564 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:31.809804 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:31.809804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:31.809805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:31.809908 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:31.809959 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:31.810157 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 15:06:31.810294 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":61}
I20260923 15:06:31.810458 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:31.803091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45265 (local address 127.0.0.1:40754)
0923 15:06:31.803265 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:31.803276 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:31.803346 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:31.803479 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:31.803480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:31.803817 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:31.803869 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.803871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.804567 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.804569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.805120 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:31.805122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:31.805467 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:31.805468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:31.805508 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:31.806085 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:31.806090 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:31.807650 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:31.809230 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.809237 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.809239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.809432 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.809652 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:31.809653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:31.809653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:31.809724 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:31.809971 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:31.809977 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:31.810053 (+ 76us) client_negotiation.cc:769] Sending connection context
0923 15:06:31.810116 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:06:31.810222 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":75,"threads_started":1}
I20260923 15:06:31.812608 9903 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:40754:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 15:06:31.812858 9903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:31.814487 10180 tablet_service.cc:1467] Processing CreateTablet for tablet 61ce14d1cf45491282822f5f3a0baa33 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a3e9894aff04e00bfa0031091b6d233]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:31.814527 10320 tablet_service.cc:1467] Processing CreateTablet for tablet 61ce14d1cf45491282822f5f3a0baa33 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a3e9894aff04e00bfa0031091b6d233]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:31.814632 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ce14d1cf45491282822f5f3a0baa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.814742 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ce14d1cf45491282822f5f3a0baa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.816037 10410 tablet_bootstrap.cc:492] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: Bootstrap starting.
I20260923 15:06:31.816176 10037 tablet_service.cc:1467] Processing CreateTablet for tablet 61ce14d1cf45491282822f5f3a0baa33 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8a3e9894aff04e00bfa0031091b6d233]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:31.816340 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61ce14d1cf45491282822f5f3a0baa33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:31.816511 10409 tablet_bootstrap.cc:492] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: Bootstrap starting.
I20260923 15:06:31.816740 10410 tablet_bootstrap.cc:654] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.817090 10409 tablet_bootstrap.cc:654] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.817514 10410 tablet_bootstrap.cc:492] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: No bootstrap required, opened a new log
I20260923 15:06:31.817564 10410 ts_tablet_manager.cc:1397] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:31.817724 10410 raft_consensus.cc:348] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:31.817786 10410 raft_consensus.cc:374] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.817806 10410 raft_consensus.cc:729] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b04974649ad477ea37705e80acd3d1f, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.817855 10410 consensus_queue.cc:260] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:31.817934 10409 tablet_bootstrap.cc:492] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: No bootstrap required, opened a new log
I20260923 15:06:31.817950 10410 ts_tablet_manager.cc:1428] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.817984 10409 ts_tablet_manager.cc:1397] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:31.818042 10417 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:31.818163 10417 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:31.818135 10409 raft_consensus.cc:348] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:31.818216 10409 raft_consensus.cc:374] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.818275 10409 raft_consensus.cc:729] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.818312 10411 tablet_bootstrap.cc:492] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: Bootstrap starting.
I20260923 15:06:31.818327 10409 consensus_queue.cc:260] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:31.818452 10409 ts_tablet_manager.cc:1428] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:31.818616 10415 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:31.818718 10415 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:31.818753 10411 tablet_bootstrap.cc:654] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:31.819522 10411 tablet_bootstrap.cc:492] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: No bootstrap required, opened a new log
I20260923 15:06:31.819571 10411 ts_tablet_manager.cc:1397] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:31.819697 10411 raft_consensus.cc:348] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:31.819759 10411 raft_consensus.cc:374] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:31.819772 10411 raft_consensus.cc:729] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Initialized, Role: FOLLOWER
I20260923 15:06:31.819828 10411 consensus_queue.cc:260] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:31.819917 10411 ts_tablet_manager.cc:1428] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:31.819955 10416 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:31.820021 10416 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: This TxnStatusTablet replica's current role is: FOLLOWER
W20260923 15:06:31.881685 10107 tablet.cc:2369] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:31.953891 10387 tablet.cc:2369] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:32.071462 10415 raft_consensus.cc:482] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:32.071568 10415 raft_consensus.cc:504] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.071980 10415 leader_election.cc:288] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467), 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:32.074887 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.072175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:37967 (local address 127.7.250.67:37067)
0923 15:06:32.072238 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:32.072250 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:32.072297 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:32.072452 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:32.072452 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:32.072454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:32.072528 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.072530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.073227 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.073228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.073974 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.073976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.074600 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.074601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.074697 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.074700 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:32.074765 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:06:32.074808 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":2}
I20260923 15:06:32.074887 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.072257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:37067 (local address 127.7.250.66:37967)
0923 15:06:32.072309 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.072311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.072323 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:32.072356 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.072356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.072366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.072446 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:32.072626 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.073113 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.074076 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.074516 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.074602 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.074611 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.074784 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.074822 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 15:06:32.074998 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.072273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:37863 (local address 127.7.250.65:46467)
0923 15:06:32.072320 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.072323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.072460 (+ 137us) server_negotiation.cc:407] Connection header received
0923 15:06:32.072544 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.072545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.072552 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.072616 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:32.072819 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.073279 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.074195 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.074605 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.074722 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.074732 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.074892 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.074938 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:06:32.074998 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.072176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:46467 (local address 127.7.250.67:37863)
0923 15:06:32.072382 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:32.072391 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:32.072469 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:32.072673 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:32.072675 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:32.072677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:32.072737 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.072738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.073364 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.073365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.074097 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.074098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.074738 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.074739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.074808 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.074811 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:32.074874 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:06:32.074915 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":20,"thread_start_us":96,"threads_started":1}
I20260923 15:06:32.075139 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ce14d1cf45491282822f5f3a0baa33" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f" is_pre_election: true
I20260923 15:06:32.075244 10200 raft_consensus.cc:2454] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 0.
I20260923 15:06:32.075256 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ce14d1cf45491282822f5f3a0baa33" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf2942fb965445db54e6f1985e89499" is_pre_election: true
I20260923 15:06:32.075368 10060 raft_consensus.cc:2454] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 0.
I20260923 15:06:32.075443 10271 leader_election.cc:302] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, fb16d0bc51814f47bbf0d1f94f323002; no voters:
I20260923 15:06:32.075595 10415 raft_consensus.cc:2781] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:32.075659 10415 raft_consensus.cc:482] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:32.075692 10415 raft_consensus.cc:3037] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:32.076390 10415 raft_consensus.cc:504] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.076540 10415 leader_election.cc:288] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 election: Requested vote from peers cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467), 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:32.076756 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ce14d1cf45491282822f5f3a0baa33" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf2942fb965445db54e6f1985e89499"
I20260923 15:06:32.076750 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61ce14d1cf45491282822f5f3a0baa33" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f"
I20260923 15:06:32.076838 10200 raft_consensus.cc:3037] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:32.076838 10060 raft_consensus.cc:3037] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:32.077586 10200 raft_consensus.cc:2454] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 1.
I20260923 15:06:32.077652 10060 raft_consensus.cc:2454] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 1.
I20260923 15:06:32.077822 10274 leader_election.cc:302] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbf2942fb965445db54e6f1985e89499, fb16d0bc51814f47bbf0d1f94f323002; no voters:
I20260923 15:06:32.077930 10415 raft_consensus.cc:2781] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:32.078078 10415 raft_consensus.cc:686] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 LEADER]: Becoming Leader. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Running, Role: LEADER
I20260923 15:06:32.078179 10415 consensus_queue.cc:237] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.078651 10431 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: TxnStatusTablet state changed. Reason: New leader fb16d0bc51814f47bbf0d1f94f323002. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.078745 10431 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:32.078994 10433 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 15:06:32.079123 10433 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 15:06:32.079089 9902 catalog_manager.cc:5495] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } health_report { overall_health: HEALTHY } } }
I20260923 15:06:32.103580 10426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.100839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.0.0.1:37060)
0923 15:06:32.100938 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:32.100955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:32.101028 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:32.101225 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:32.101225 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:32.101227 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:32.101286 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.101288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.102001 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.102002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.102647 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.102649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.103032 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.103033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.103095 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.103114 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.103396 (+ 282us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.103405 (+ 9us) client_negotiation.cc:769] Sending connection context
0923 15:06:32.103450 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 15:06:32.103498 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260923 15:06:32.103577 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.100954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.7.250.67:42811)
0923 15:06:32.101023 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.101026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.101036 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:32.101111 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.101112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.101135 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.101219 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:32.101358 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.101894 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.102733 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.102927 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.103015 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.103213 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.103321 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.103406 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.103453 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.103475 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:06:32.105796 10200 raft_consensus.cc:1261] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Refusing update from remote peer fb16d0bc51814f47bbf0d1f94f323002: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:32.105794 10060 raft_consensus.cc:1261] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Refusing update from remote peer fb16d0bc51814f47bbf0d1f94f323002: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:32.106168 10431 consensus_queue.cc:1035] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:32.106331 10415 consensus_queue.cc:1035] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:32.107478 10416 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: TxnStatusTablet state changed. Reason: New leader fb16d0bc51814f47bbf0d1f94f323002. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.107507 10417 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: TxnStatusTablet state changed. Reason: New leader fb16d0bc51814f47bbf0d1f94f323002. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.107576 10416 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:32.107594 10417 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:32.108734 10415 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.108824 10415 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:32.108889 10437 mvcc.cc:205] Tried to move back new op lower bound from 7332560863662239744 to 7332560863552782336. Current Snapshot: MvccSnapshot[applied={T|T < 7332560863662239744}]
I20260923 15:06:32.108911 10436 mvcc.cc:205] Tried to move back new op lower bound from 7332560863662239744 to 7332560863552782336. Current Snapshot: MvccSnapshot[applied={T|T < 7332560863662239744}]
I20260923 15:06:32.109037 10417 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.109107 10417 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P 5b04974649ad477ea37705e80acd3d1f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:32.109330 10415 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.109395 10415 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P fb16d0bc51814f47bbf0d1f94f323002: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:32.109787 10416 tablet_replica.cc:439] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } } }
I20260923 15:06:32.109859 10416 tablet_replica.cc:442] T 61ce14d1cf45491282822f5f3a0baa33 P cbf2942fb965445db54e6f1985e89499: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:32.111039 10438 mvcc.cc:205] Tried to move back new op lower bound from 7332560863662239744 to 7332560863552782336. Current Snapshot: MvccSnapshot[applied={T|T < 7332560863662239744 or (T in {7332560863662239744})}]
I20260923 15:06:32.113617 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.111200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.7.250.65:46467)
0923 15:06:32.111254 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.111256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.111263 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:32.111293 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.111294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.111318 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.111391 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:32.111549 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.112025 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.112794 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.113015 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.113102 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.113282 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.113385 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.113476 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.113502 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.113523 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:06:32.118856 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.116196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:54025 (local address 127.7.250.67:42811)
0923 15:06:32.116236 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.116238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.116353 (+ 115us) server_negotiation.cc:407] Connection header received
0923 15:06:32.116427 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.116428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.116436 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.116481 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:32.116611 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.117042 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.118176 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.118401 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.118456 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.118660 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.118686 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.118724 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.118783 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.118800 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:06:32.118914 10447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.116146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.7.250.65:54025)
0923 15:06:32.116302 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:32.116313 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:32.116369 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:32.116490 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:32.116491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:32.116492 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:32.116551 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.116553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.117123 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.117124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.118081 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.118084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.118475 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.118476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.118544 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.118560 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.118730 (+ 170us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.118731 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:32.118781 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 15:06:32.118825 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260923 15:06:32.121553 10340 raft_consensus.cc:1261] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Refusing update from remote peer cbf2942fb965445db54e6f1985e89499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:32.121730 10200 raft_consensus.cc:1261] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Refusing update from remote peer cbf2942fb965445db54e6f1985e89499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:32.121974 10419 consensus_queue.cc:1035] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:32.122112 10416 consensus_queue.cc:1035] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:32.131254 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.128674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:33495 (local address 127.7.250.65:46467)
0923 15:06:32.128710 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.128712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.128748 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:06:32.128810 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.128811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.128819 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.128903 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:32.129055 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.129536 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.130649 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.130831 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.130884 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.131074 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.131098 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.131131 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.131182 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.131199 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:06:32.131301 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.128642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:46467 (local address 127.7.250.67:33495)
0923 15:06:32.128701 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:32.128713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:32.128770 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:32.128913 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:32.128914 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:32.128915 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:32.129000 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.129001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.129602 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.129603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.130565 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.130567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.130902 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.130903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.130976 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.130999 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.131132 (+ 133us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.131134 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:32.131181 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:06:32.131223 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 15:06:32.140434 9902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:32.140560 9902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:32.141345 9902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Sending DeleteTablet for 3 replicas of tablet 412a74d54d6c4ba1b4998941fa4ee692
I20260923 15:06:32.141675 10037 tablet_service.cc:1514] Processing DeleteTablet for tablet 412a74d54d6c4ba1b4998941fa4ee692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:32 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32996
I20260923 15:06:32.141731 10320 tablet_service.cc:1514] Processing DeleteTablet for tablet 412a74d54d6c4ba1b4998941fa4ee692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:32 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37044
I20260923 15:06:32.141928 10455 tablet_replica.cc:330] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: stopping tablet replica
I20260923 15:06:32.142040 10455 raft_consensus.cc:2229] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:32.142115 10180 tablet_service.cc:1514] Processing DeleteTablet for tablet 412a74d54d6c4ba1b4998941fa4ee692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:32 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:34236
I20260923 15:06:32.142194 10455 raft_consensus.cc:2258] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:32.142596 9902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 15:06:32.142624 10455 ts_tablet_manager.cc:1905] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:06:32.142732 9902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:32.144019 10037 tablet_service.cc:1467] Processing CreateTablet for tablet 2244f7c4c20141328c91cb30a373d569 (DEFAULT_TABLE table=test-table [id=6f7f9f31083e42d5a09de35face3f0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:32.144130 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2244f7c4c20141328c91cb30a373d569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:32.144629 10455 ts_tablet_manager.cc:1918] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:32.144697 10455 log.cc:1196] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/412a74d54d6c4ba1b4998941fa4ee692
I20260923 15:06:32.144970 10455 ts_tablet_manager.cc:1939] T 412a74d54d6c4ba1b4998941fa4ee692 P cbf2942fb965445db54e6f1985e89499: Deleting consensus metadata
I20260923 15:06:32.145318 10411 tablet_bootstrap.cc:492] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Bootstrap starting.
I20260923 15:06:32.145370 9890 catalog_manager.cc:4841] TS cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467): tablet 412a74d54d6c4ba1b4998941fa4ee692 (table test-table [id=30a1d4c4b4a341d3a86d37034c72ce47]) successfully deleted
I20260923 15:06:32.145685 10411 tablet_bootstrap.cc:654] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:32.146466 10457 tablet_replica.cc:330] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: stopping tablet replica
I20260923 15:06:32.146530 10320 tablet_service.cc:1467] Processing CreateTablet for tablet 2244f7c4c20141328c91cb30a373d569 (DEFAULT_TABLE table=test-table [id=6f7f9f31083e42d5a09de35face3f0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:32.146606 10457 raft_consensus.cc:2229] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:32.146657 10320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2244f7c4c20141328c91cb30a373d569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:32.146668 10457 raft_consensus.cc:2258] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:32.146833 10456 tablet_replica.cc:330] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: stopping tablet replica
I20260923 15:06:32.146927 10456 raft_consensus.cc:2229] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:32.146978 10456 raft_consensus.cc:2258] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:32.147178 10457 ts_tablet_manager.cc:1905] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:32.148135 10409 tablet_bootstrap.cc:492] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Bootstrap starting.
I20260923 15:06:32.148428 10456 ts_tablet_manager.cc:1905] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:32.148502 10409 tablet_bootstrap.cc:654] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:32.148944 10457 ts_tablet_manager.cc:1918] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:32.149010 10457 log.cc:1196] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/412a74d54d6c4ba1b4998941fa4ee692
I20260923 15:06:32.149247 10457 ts_tablet_manager.cc:1939] T 412a74d54d6c4ba1b4998941fa4ee692 P 5b04974649ad477ea37705e80acd3d1f: Deleting consensus metadata
I20260923 15:06:32.149501 10411 tablet_bootstrap.cc:492] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: No bootstrap required, opened a new log
I20260923 15:06:32.149552 10411 ts_tablet_manager.cc:1397] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 15:06:32.149693 10411 raft_consensus.cc:348] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.149757 10411 raft_consensus.cc:374] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:32.149775 10411 raft_consensus.cc:729] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Initialized, Role: FOLLOWER
I20260923 15:06:32.149827 10411 consensus_queue.cc:260] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.149935 10411 ts_tablet_manager.cc:1428] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:32.150873 10409 tablet_bootstrap.cc:492] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: No bootstrap required, opened a new log
I20260923 15:06:32.150888 10456 ts_tablet_manager.cc:1918] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:32.150915 10409 ts_tablet_manager.cc:1397] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 15:06:32.150954 10456 log.cc:1196] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/412a74d54d6c4ba1b4998941fa4ee692
I20260923 15:06:32.151077 10409 raft_consensus.cc:348] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.151132 10409 raft_consensus.cc:374] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:32.151149 10409 raft_consensus.cc:729] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Initialized, Role: FOLLOWER
I20260923 15:06:32.151185 10409 consensus_queue.cc:260] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.151216 10456 ts_tablet_manager.cc:1939] T 412a74d54d6c4ba1b4998941fa4ee692 P fb16d0bc51814f47bbf0d1f94f323002: Deleting consensus metadata
I20260923 15:06:32.151253 10409 ts_tablet_manager.cc:1428] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:32.151597 9887 catalog_manager.cc:4841] TS fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811): tablet 412a74d54d6c4ba1b4998941fa4ee692 (table test-table [id=30a1d4c4b4a341d3a86d37034c72ce47]) successfully deleted
I20260923 15:06:32.150342 10180 tablet_service.cc:1467] Processing CreateTablet for tablet 2244f7c4c20141328c91cb30a373d569 (DEFAULT_TABLE table=test-table [id=6f7f9f31083e42d5a09de35face3f0ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:32.162428 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2244f7c4c20141328c91cb30a373d569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:32.162869 9887 catalog_manager.cc:4841] TS 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967): tablet 412a74d54d6c4ba1b4998941fa4ee692 (table test-table [id=30a1d4c4b4a341d3a86d37034c72ce47]) successfully deleted
I20260923 15:06:32.163952 10410 tablet_bootstrap.cc:492] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Bootstrap starting.
I20260923 15:06:32.163980 10448 raft_consensus.cc:482] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:32.164055 10448 raft_consensus.cc:504] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.164194 10448 leader_election.cc:288] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467), 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:32.164393 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2244f7c4c20141328c91cb30a373d569" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f" is_pre_election: true
W20260923 15:06:32.164752 10271 leader_election.cc:341] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967): Illegal state: must be running to vote when last-logged opid is not known
I20260923 15:06:32.164846 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2244f7c4c20141328c91cb30a373d569" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf2942fb965445db54e6f1985e89499" is_pre_election: true
I20260923 15:06:32.164935 10060 raft_consensus.cc:2454] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 0.
I20260923 15:06:32.164417 10410 tablet_bootstrap.cc:654] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:32.165117 10274 leader_election.cc:302] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cbf2942fb965445db54e6f1985e89499, fb16d0bc51814f47bbf0d1f94f323002; no voters: 5b04974649ad477ea37705e80acd3d1f
I20260923 15:06:32.165234 10448 raft_consensus.cc:2781] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:32.165261 10448 raft_consensus.cc:482] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:32.165282 10448 raft_consensus.cc:3037] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:32.166055 10410 tablet_bootstrap.cc:492] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: No bootstrap required, opened a new log
I20260923 15:06:32.166102 10410 ts_tablet_manager.cc:1397] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:32.166100 10448 raft_consensus.cc:504] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.166239 10448 leader_election.cc:288] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 election: Requested vote from peers cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467), 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967)
I20260923 15:06:32.166337 10410 raft_consensus.cc:348] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.166410 10410 raft_consensus.cc:374] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:32.166432 10410 raft_consensus.cc:729] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b04974649ad477ea37705e80acd3d1f, State: Initialized, Role: FOLLOWER
I20260923 15:06:32.166472 10410 consensus_queue.cc:260] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.166482 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2244f7c4c20141328c91cb30a373d569" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f"
I20260923 15:06:32.166551 10200 raft_consensus.cc:3037] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:32.166563 10410 ts_tablet_manager.cc:1428] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:32.167219 10200 raft_consensus.cc:2454] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 1.
I20260923 15:06:32.167479 10271 leader_election.cc:302] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, fb16d0bc51814f47bbf0d1f94f323002; no voters:
I20260923 15:06:32.167685 10060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2244f7c4c20141328c91cb30a373d569" candidate_uuid: "fb16d0bc51814f47bbf0d1f94f323002" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf2942fb965445db54e6f1985e89499"
I20260923 15:06:32.167766 10060 raft_consensus.cc:3037] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:32.168537 10060 raft_consensus.cc:2454] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb16d0bc51814f47bbf0d1f94f323002 in term 1.
I20260923 15:06:32.168941 10448 raft_consensus.cc:2781] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:32.169008 10448 raft_consensus.cc:686] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 LEADER]: Becoming Leader. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Running, Role: LEADER
I20260923 15:06:32.169065 10448 consensus_queue.cc:237] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:32.169620 9902 catalog_manager.cc:5495] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "fb16d0bc51814f47bbf0d1f94f323002" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } health_report { overall_health: HEALTHY } } }
I20260923 15:06:32.181929 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.178931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37070 (local address 127.7.250.67:42811)
0923 15:06:32.178974 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.178976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.178987 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:32.179067 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.179068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.179080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.179139 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:32.179260 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.180023 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.181126 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.181427 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.181498 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.181666 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.181704 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.181764 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.181824 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.181849 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:32.186987 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.184172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.7.250.67:39505)
0923 15:06:32.184233 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:32.184243 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:32.184278 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:32.184400 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:32.184401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:32.184402 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:32.184441 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.184442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.185281 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.185282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.186136 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:32.186138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.186628 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.186628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:32.186671 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.186689 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.186834 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.186835 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:32.186872 (+ 37us) client_negotiation.cc:240] Negotiation successful
0923 15:06:32.186912 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:06:32.187413 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:32.184212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:39505 (local address 127.7.250.67:42811)
0923 15:06:32.184245 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:32.184246 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:32.184269 (+ 23us) server_negotiation.cc:407] Connection header received
0923 15:06:32.184330 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:32.184331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:32.184341 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:32.184400 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:32.184491 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.185189 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.186239 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:32.186547 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:32.186602 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:32.186752 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:32.186780 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:32.186818 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:32.187338 (+ 520us) server_negotiation.cc:299] Negotiation successful
0923 15:06:32.187355 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 15:06:32.190848 10200 raft_consensus.cc:1261] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Refusing update from remote peer fb16d0bc51814f47bbf0d1f94f323002: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:32.190882 10060 raft_consensus.cc:1261] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Refusing update from remote peer fb16d0bc51814f47bbf0d1f94f323002: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:32.191071 10415 consensus_queue.cc:1035] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:32.191182 10448 consensus_queue.cc:1035] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:06:32.196694 10436 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 15:06:33.719915 9902 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:33.720063 9902 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:33.721115 9902 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Sending DeleteTablet for 3 replicas of tablet 2244f7c4c20141328c91cb30a373d569
I20260923 15:06:33.721410 10037 tablet_service.cc:1514] Processing DeleteTablet for tablet 2244f7c4c20141328c91cb30a373d569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:33 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32996
I20260923 15:06:33.721444 10180 tablet_service.cc:1514] Processing DeleteTablet for tablet 2244f7c4c20141328c91cb30a373d569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:33 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:34236
I20260923 15:06:33.721726 10319 tablet_service.cc:1514] Processing DeleteTablet for tablet 2244f7c4c20141328c91cb30a373d569 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:33 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37044
I20260923 15:06:33.721757 10485 tablet_replica.cc:330] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: stopping tablet replica
I20260923 15:06:33.721823 10485 raft_consensus.cc:2229] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:33.721907 10485 raft_consensus.cc:2258] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:33.722193 10485 ts_tablet_manager.cc:1905] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:33.723539 10484 tablet_replica.cc:330] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: stopping tablet replica
I20260923 15:06:33.723675 10484 raft_consensus.cc:2229] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:33.723773 10484 raft_consensus.cc:2258] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:33.724087 10486 tablet_replica.cc:330] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: stopping tablet replica
I20260923 15:06:33.724102 10484 ts_tablet_manager.cc:1905] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:33.724202 10486 raft_consensus.cc:2229] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:33.724335 10486 raft_consensus.cc:2258] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:33.724466 10485 ts_tablet_manager.cc:1918] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:06:33.724529 10485 log.cc:1196] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/2244f7c4c20141328c91cb30a373d569
I20260923 15:06:33.724589 10486 ts_tablet_manager.cc:1905] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:33.724812 10485 ts_tablet_manager.cc:1939] T 2244f7c4c20141328c91cb30a373d569 P 5b04974649ad477ea37705e80acd3d1f: Deleting consensus metadata
I20260923 15:06:33.725138 9887 catalog_manager.cc:4841] TS 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967): tablet 2244f7c4c20141328c91cb30a373d569 (table test-table [id=6f7f9f31083e42d5a09de35face3f0ba]) successfully deleted
I20260923 15:06:33.725819 10484 ts_tablet_manager.cc:1918] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:06:33.725886 10484 log.cc:1196] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/2244f7c4c20141328c91cb30a373d569
I20260923 15:06:33.726127 10484 ts_tablet_manager.cc:1939] T 2244f7c4c20141328c91cb30a373d569 P cbf2942fb965445db54e6f1985e89499: Deleting consensus metadata
I20260923 15:06:33.726130 10486 ts_tablet_manager.cc:1918] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:06:33.726193 10486 log.cc:1196] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/2244f7c4c20141328c91cb30a373d569
I20260923 15:06:33.726450 10486 ts_tablet_manager.cc:1939] T 2244f7c4c20141328c91cb30a373d569 P fb16d0bc51814f47bbf0d1f94f323002: Deleting consensus metadata
I20260923 15:06:33.726529 9890 catalog_manager.cc:4841] TS cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467): tablet 2244f7c4c20141328c91cb30a373d569 (table test-table [id=6f7f9f31083e42d5a09de35face3f0ba]) successfully deleted
I20260923 15:06:33.726897 9887 catalog_manager.cc:4841] TS fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811): tablet 2244f7c4c20141328c91cb30a373d569 (table test-table [id=6f7f9f31083e42d5a09de35face3f0ba]) successfully deleted
Sep 23 15:06:33 dist-test-slave-bb3x krb5kdc[9858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:06:33 dist-test-slave-bb3x krb5kdc[9858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175993, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:33.748294 10494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:33.740513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57442 (local address 127.7.250.126:45265)
0923 15:06:33.740679 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:33.740681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:33.740690 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:33.740717 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:33.740717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:33.740728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:33.740790 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:33.741494 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.742036 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.742894 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.743123 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.743182 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:33.745705 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:33.745707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:33.745707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:33.745714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:33.747247 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:33.747595 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:33.747596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:33.747596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:33.747657 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:33.747831 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:33.747832 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:33.747832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:33.747935 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:33.748009 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:33.748120 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 15:06:33.748182 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":108,"threads_started":1}
I20260923 15:06:33.749943 9902 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:33.750136 9902 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:33.751899 10180 tablet_service.cc:1467] Processing CreateTablet for tablet b1817a564c3f49b792d337fa77e64cfd (DEFAULT_TABLE table=test-table [id=63e7a1e20a3a470497174c52d349a71e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:33.751973 10037 tablet_service.cc:1467] Processing CreateTablet for tablet b1817a564c3f49b792d337fa77e64cfd (DEFAULT_TABLE table=test-table [id=63e7a1e20a3a470497174c52d349a71e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:33.752082 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1817a564c3f49b792d337fa77e64cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:33.752106 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1817a564c3f49b792d337fa77e64cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:33.753695 10495 tablet_bootstrap.cc:492] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Bootstrap starting.
I20260923 15:06:33.753732 10319 tablet_service.cc:1467] Processing CreateTablet for tablet b1817a564c3f49b792d337fa77e64cfd (DEFAULT_TABLE table=test-table [id=63e7a1e20a3a470497174c52d349a71e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:33.753894 10319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1817a564c3f49b792d337fa77e64cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:33.754132 10495 tablet_bootstrap.cc:654] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:33.754429 10496 tablet_bootstrap.cc:492] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Bootstrap starting.
I20260923 15:06:33.754807 10495 tablet_bootstrap.cc:492] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: No bootstrap required, opened a new log
I20260923 15:06:33.754859 10496 tablet_bootstrap.cc:654] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:33.754875 10495 ts_tablet_manager.cc:1397] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:33.755026 10495 raft_consensus.cc:348] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.755110 10495 raft_consensus.cc:374] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:33.755136 10495 raft_consensus.cc:729] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b04974649ad477ea37705e80acd3d1f, State: Initialized, Role: FOLLOWER
I20260923 15:06:33.755179 10495 consensus_queue.cc:260] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.755360 10495 ts_tablet_manager.cc:1428] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:33.756035 10500 tablet_bootstrap.cc:492] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Bootstrap starting.
I20260923 15:06:33.756325 10496 tablet_bootstrap.cc:492] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: No bootstrap required, opened a new log
I20260923 15:06:33.756398 10500 tablet_bootstrap.cc:654] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:33.756417 10496 ts_tablet_manager.cc:1397] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:33.756536 10496 raft_consensus.cc:348] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.756641 10496 raft_consensus.cc:374] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:33.756666 10496 raft_consensus.cc:729] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Initialized, Role: FOLLOWER
I20260923 15:06:33.756711 10496 consensus_queue.cc:260] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.756916 10496 ts_tablet_manager.cc:1428] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:33.757083 10500 tablet_bootstrap.cc:492] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: No bootstrap required, opened a new log
I20260923 15:06:33.757138 10500 ts_tablet_manager.cc:1397] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:33.757259 10500 raft_consensus.cc:348] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.757350 10500 raft_consensus.cc:374] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:33.757373 10500 raft_consensus.cc:729] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Initialized, Role: FOLLOWER
I20260923 15:06:33.757417 10500 consensus_queue.cc:260] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.757500 10500 ts_tablet_manager.cc:1428] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:33.761337 10502 raft_consensus.cc:482] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:33.761408 10502 raft_consensus.cc:504] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.761539 10502 leader_election.cc:288] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967), fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811)
I20260923 15:06:33.761724 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1817a564c3f49b792d337fa77e64cfd" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f" is_pre_election: true
I20260923 15:06:33.761723 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1817a564c3f49b792d337fa77e64cfd" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb16d0bc51814f47bbf0d1f94f323002" is_pre_election: true
I20260923 15:06:33.761821 10200 raft_consensus.cc:2454] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 0.
I20260923 15:06:33.761848 10340 raft_consensus.cc:2454] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 0.
I20260923 15:06:33.762015 9991 leader_election.cc:302] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, cbf2942fb965445db54e6f1985e89499; no voters:
I20260923 15:06:33.762137 10502 raft_consensus.cc:2781] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:33.762219 10502 raft_consensus.cc:482] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:33.762241 10502 raft_consensus.cc:3037] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:33.762989 10502 raft_consensus.cc:504] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.763089 10502 leader_election.cc:288] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 election: Requested vote from peers 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967), fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811)
I20260923 15:06:33.763262 10200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1817a564c3f49b792d337fa77e64cfd" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b04974649ad477ea37705e80acd3d1f"
I20260923 15:06:33.763335 10200 raft_consensus.cc:3037] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:33.763295 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1817a564c3f49b792d337fa77e64cfd" candidate_uuid: "cbf2942fb965445db54e6f1985e89499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb16d0bc51814f47bbf0d1f94f323002"
I20260923 15:06:33.763389 10340 raft_consensus.cc:3037] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:33.763923 10200 raft_consensus.cc:2454] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 1.
I20260923 15:06:33.764071 10340 raft_consensus.cc:2454] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cbf2942fb965445db54e6f1985e89499 in term 1.
I20260923 15:06:33.764174 9991 leader_election.cc:302] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, cbf2942fb965445db54e6f1985e89499; no voters:
I20260923 15:06:33.764324 10502 raft_consensus.cc:2781] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:33.764504 10502 raft_consensus.cc:686] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 1 LEADER]: Becoming Leader. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Running, Role: LEADER
I20260923 15:06:33.764611 10502 consensus_queue.cc:237] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.765172 9903 catalog_manager.cc:5495] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 reported cstate change: term changed from 0 to 1, leader changed from <none> to cbf2942fb965445db54e6f1985e89499 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "cbf2942fb965445db54e6f1985e89499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:33.774791 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:33.772249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.7.250.65:46467)
0923 15:06:33.772417 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:33.772419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:33.772425 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:06:33.772443 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:33.772444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:33.772452 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:33.772537 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:33.772714 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.773228 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.774034 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.774305 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.774366 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:33.774523 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:33.774557 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:33.774610 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:33.774672 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 15:06:33.774700 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260923 15:06:33.777154 10340 raft_consensus.cc:1261] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Refusing update from remote peer cbf2942fb965445db54e6f1985e89499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:33.777206 10200 raft_consensus.cc:1261] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Refusing update from remote peer cbf2942fb965445db54e6f1985e89499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:33.777395 10503 consensus_queue.cc:1035] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:33.777513 10502 consensus_queue.cc:1035] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:33.787271 9903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:33.787362 9903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:33.787935 9903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Sending DeleteTablet for 3 replicas of tablet b1817a564c3f49b792d337fa77e64cfd
I20260923 15:06:33.788276 10180 tablet_service.cc:1514] Processing DeleteTablet for tablet b1817a564c3f49b792d337fa77e64cfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:33 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:34236
I20260923 15:06:33.788265 10319 tablet_service.cc:1514] Processing DeleteTablet for tablet b1817a564c3f49b792d337fa77e64cfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:33 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37044
I20260923 15:06:33.788362 10485 tablet_replica.cc:330] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: stopping tablet replica
I20260923 15:06:33.788375 10486 tablet_replica.cc:330] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: stopping tablet replica
I20260923 15:06:33.788412 10485 raft_consensus.cc:2229] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:33.788448 10485 raft_consensus.cc:2258] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:33.788443 10486 raft_consensus.cc:2229] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:33.788480 10486 raft_consensus.cc:2258] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:33.788518 10037 tablet_service.cc:1514] Processing DeleteTablet for tablet b1817a564c3f49b792d337fa77e64cfd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:33 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32996
I20260923 15:06:33.788476 9903 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 15:06:33.788610 10484 tablet_replica.cc:330] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: stopping tablet replica
W20260923 15:06:33.788616 9903 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:33.788661 10484 raft_consensus.cc:2229] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:33.788730 10484 raft_consensus.cc:2258] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:33.788805 10485 ts_tablet_manager.cc:1905] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:33.788805 10486 ts_tablet_manager.cc:1905] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:33.789165 10484 ts_tablet_manager.cc:1905] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:33.790601 10485 ts_tablet_manager.cc:1918] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:33.790654 10485 log.cc:1196] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/b1817a564c3f49b792d337fa77e64cfd
I20260923 15:06:33.790864 10484 ts_tablet_manager.cc:1918] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:33.790879 10485 ts_tablet_manager.cc:1939] T b1817a564c3f49b792d337fa77e64cfd P 5b04974649ad477ea37705e80acd3d1f: Deleting consensus metadata
I20260923 15:06:33.790913 10484 log.cc:1196] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/b1817a564c3f49b792d337fa77e64cfd
I20260923 15:06:33.791113 10484 ts_tablet_manager.cc:1939] T b1817a564c3f49b792d337fa77e64cfd P cbf2942fb965445db54e6f1985e89499: Deleting consensus metadata
I20260923 15:06:33.791225 9887 catalog_manager.cc:4841] TS 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967): tablet b1817a564c3f49b792d337fa77e64cfd (table test-table [id=63e7a1e20a3a470497174c52d349a71e]) successfully deleted
I20260923 15:06:33.791481 10486 ts_tablet_manager.cc:1918] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:33.791540 10486 log.cc:1196] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/b1817a564c3f49b792d337fa77e64cfd
I20260923 15:06:33.791640 10037 tablet_service.cc:1467] Processing CreateTablet for tablet b7cfca77f7214d23b9f473ef4ed11a34 (DEFAULT_TABLE table=test-table [id=823d32beff2b46689f79293ded474154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:33.791745 10037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cfca77f7214d23b9f473ef4ed11a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:33.791755 10319 tablet_service.cc:1467] Processing CreateTablet for tablet b7cfca77f7214d23b9f473ef4ed11a34 (DEFAULT_TABLE table=test-table [id=823d32beff2b46689f79293ded474154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:33.791790 10486 ts_tablet_manager.cc:1939] T b1817a564c3f49b792d337fa77e64cfd P fb16d0bc51814f47bbf0d1f94f323002: Deleting consensus metadata
I20260923 15:06:33.791847 10319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cfca77f7214d23b9f473ef4ed11a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:33.792095 9887 catalog_manager.cc:4841] TS fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811): tablet b1817a564c3f49b792d337fa77e64cfd (table test-table [id=63e7a1e20a3a470497174c52d349a71e]) successfully deleted
I20260923 15:06:33.792701 10180 tablet_service.cc:1467] Processing CreateTablet for tablet b7cfca77f7214d23b9f473ef4ed11a34 (DEFAULT_TABLE table=test-table [id=823d32beff2b46689f79293ded474154]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:33.792724 10496 tablet_bootstrap.cc:492] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499: Bootstrap starting.
I20260923 15:06:33.792836 10180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7cfca77f7214d23b9f473ef4ed11a34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:33.792968 10500 tablet_bootstrap.cc:492] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Bootstrap starting.
I20260923 15:06:33.793022 10496 tablet_bootstrap.cc:654] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:33.793308 10500 tablet_bootstrap.cc:654] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:33.794064 10495 tablet_bootstrap.cc:492] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Bootstrap starting.
I20260923 15:06:33.794235 9890 catalog_manager.cc:4841] TS cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467): tablet b1817a564c3f49b792d337fa77e64cfd (table test-table [id=63e7a1e20a3a470497174c52d349a71e]) successfully deleted
I20260923 15:06:33.794657 10495 tablet_bootstrap.cc:654] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:33.795197 10500 tablet_bootstrap.cc:492] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: No bootstrap required, opened a new log
I20260923 15:06:33.795217 10496 tablet_bootstrap.cc:492] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499: No bootstrap required, opened a new log
I20260923 15:06:33.795266 10500 ts_tablet_manager.cc:1397] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:33.795290 10496 ts_tablet_manager.cc:1397] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:06:33.795390 10500 raft_consensus.cc:348] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.795411 10496 raft_consensus.cc:348] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.795471 10496 raft_consensus.cc:374] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:33.795476 10500 raft_consensus.cc:374] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:33.795500 10500 raft_consensus.cc:729] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb16d0bc51814f47bbf0d1f94f323002, State: Initialized, Role: FOLLOWER
I20260923 15:06:33.795485 10496 raft_consensus.cc:729] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbf2942fb965445db54e6f1985e89499, State: Initialized, Role: FOLLOWER
I20260923 15:06:33.795548 10500 consensus_queue.cc:260] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.795571 10496 consensus_queue.cc:260] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.795639 10500 ts_tablet_manager.cc:1428] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:33.795667 10496 ts_tablet_manager.cc:1428] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:33.795838 10495 tablet_bootstrap.cc:492] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: No bootstrap required, opened a new log
I20260923 15:06:33.795888 10495 ts_tablet_manager.cc:1397] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:06:33.796026 10495 raft_consensus.cc:348] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.796115 10495 raft_consensus.cc:374] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:33.796135 10495 raft_consensus.cc:729] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b04974649ad477ea37705e80acd3d1f, State: Initialized, Role: FOLLOWER
I20260923 15:06:33.796178 10495 consensus_queue.cc:260] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.796262 10495 ts_tablet_manager.cc:1428] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:33.994125 10499 raft_consensus.cc:482] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:33.994306 10499 raft_consensus.cc:504] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:33.994773 10499 leader_election.cc:288] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467), fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811)
I20260923 15:06:33.998497 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:33.995073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:49443 (local address 127.7.250.67:42811)
0923 15:06:33.995227 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:33.995230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:33.995273 (+ 43us) server_negotiation.cc:407] Connection header received
0923 15:06:33.995525 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:33.995526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:33.995538 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:33.995619 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:33.995861 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.996659 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.997673 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.998059 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.998168 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:33.998178 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:33.998369 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 15:06:33.998414 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":74,"threads_started":1}
I20260923 15:06:33.998521 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:33.995010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.7.250.66:49443)
0923 15:06:33.995208 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:33.995220 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:33.995292 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:33.995686 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:33.995687 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:33.995690 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:33.995769 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:33.995771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.996739 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.996740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:33.997564 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:33.997566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.998167 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.998168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:33.998247 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:33.998250 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:33.998359 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:06:33.998410 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":27,"thread_start_us":153,"threads_started":1}
I20260923 15:06:33.998734 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:33.995087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:44879 (local address 127.7.250.65:46467)
0923 15:06:33.995146 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:33.995148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:33.995251 (+ 103us) server_negotiation.cc:407] Connection header received
0923 15:06:33.995359 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:33.995360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:33.995371 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:33.995456 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:33.995778 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.996654 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.997720 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.998139 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.998310 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:33.998325 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:33.998599 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 15:06:33.998657 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:06:33.998802 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cfca77f7214d23b9f473ef4ed11a34" candidate_uuid: "5b04974649ad477ea37705e80acd3d1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb16d0bc51814f47bbf0d1f94f323002" is_pre_election: true
I20260923 15:06:33.998922 10340 raft_consensus.cc:2454] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b04974649ad477ea37705e80acd3d1f in term 0.
I20260923 15:06:33.998967 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:33.995002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:46467 (local address 127.7.250.66:44879)
0923 15:06:33.995152 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:33.995163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:33.995286 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:33.995470 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:33.995471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:33.995473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:33.995682 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:33.995685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.996804 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.996805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:33.997612 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:33.997614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:33.998304 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:33.998305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:33.998401 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:33.998405 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:33.998818 (+ 413us) client_negotiation.cc:240] Negotiation successful
0923 15:06:33.998863 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":3,"thread_start_us":79,"threads_started":1}
I20260923 15:06:33.999228 10131 leader_election.cc:302] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, fb16d0bc51814f47bbf0d1f94f323002; no voters:
I20260923 15:06:33.999279 10059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cfca77f7214d23b9f473ef4ed11a34" candidate_uuid: "5b04974649ad477ea37705e80acd3d1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf2942fb965445db54e6f1985e89499" is_pre_election: true
I20260923 15:06:33.999372 10059 raft_consensus.cc:2454] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b04974649ad477ea37705e80acd3d1f in term 0.
I20260923 15:06:33.999374 10499 raft_consensus.cc:2781] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:33.999408 10499 raft_consensus.cc:482] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:33.999431 10499 raft_consensus.cc:3037] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:34.000180 10499 raft_consensus.cc:504] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:34.000327 10499 leader_election.cc:288] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [CANDIDATE]: Term 1 election: Requested vote from peers cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467), fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811)
I20260923 15:06:34.000470 10059 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cfca77f7214d23b9f473ef4ed11a34" candidate_uuid: "5b04974649ad477ea37705e80acd3d1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbf2942fb965445db54e6f1985e89499"
I20260923 15:06:34.000545 10059 raft_consensus.cc:3037] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:34.000504 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7cfca77f7214d23b9f473ef4ed11a34" candidate_uuid: "5b04974649ad477ea37705e80acd3d1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb16d0bc51814f47bbf0d1f94f323002"
I20260923 15:06:34.000587 10340 raft_consensus.cc:3037] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:34.001353 10340 raft_consensus.cc:2454] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b04974649ad477ea37705e80acd3d1f in term 1.
I20260923 15:06:34.001353 10059 raft_consensus.cc:2454] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b04974649ad477ea37705e80acd3d1f in term 1.
I20260923 15:06:34.001529 10131 leader_election.cc:302] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b04974649ad477ea37705e80acd3d1f, fb16d0bc51814f47bbf0d1f94f323002; no voters:
I20260923 15:06:34.001648 10499 raft_consensus.cc:2781] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:34.001798 10499 raft_consensus.cc:686] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 1 LEADER]: Becoming Leader. State: Replica: 5b04974649ad477ea37705e80acd3d1f, State: Running, Role: LEADER
I20260923 15:06:34.001858 10499 consensus_queue.cc:237] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } }
I20260923 15:06:34.002628 9903 catalog_manager.cc:5495] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b04974649ad477ea37705e80acd3d1f (127.7.250.66). New cstate: current_term: 1 leader_uuid: "5b04974649ad477ea37705e80acd3d1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b04974649ad477ea37705e80acd3d1f" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 37967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:34.027360 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:34.024613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.7.250.66:37967)
0923 15:06:34.024859 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:34.024861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:34.024870 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:34.024908 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:34.024908 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:34.024929 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:34.025011 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:34.025118 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.025658 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.026435 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.026733 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.026797 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:34.026981 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:34.027097 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:34.027157 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:34.027197 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 15:06:34.027215 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":129,"threads_started":1}
I20260923 15:06:34.032747 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:34.030324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:45463 (local address 127.7.250.67:42811)
0923 15:06:34.030392 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:34.030396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:34.030408 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:34.030430 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:34.030431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:34.030441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:34.030487 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:34.030734 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.031203 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.032058 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.032262 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.032317 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:34.032496 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:34.032521 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:34.032561 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:34.032663 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 15:06:34.032687 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:06:34.032778 10522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:34.030107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42811 (local address 127.7.250.66:45463)
0923 15:06:34.030321 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:34.030332 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:34.030378 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:34.030507 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:34.030507 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:34.030509 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:34.030623 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:34.030625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.031283 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.031284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:34.031964 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:34.031967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.032354 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.032355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:34.032404 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:34.032417 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:34.032618 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:34.032620 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:34.032660 (+ 40us) client_negotiation.cc:240] Negotiation successful
0923 15:06:34.032698 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260923 15:06:34.035102 10340 raft_consensus.cc:1261] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Refusing update from remote peer 5b04974649ad477ea37705e80acd3d1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:34.035167 10059 raft_consensus.cc:1261] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Refusing update from remote peer 5b04974649ad477ea37705e80acd3d1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:34.035406 10517 consensus_queue.cc:1035] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb16d0bc51814f47bbf0d1f94f323002" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:34.035557 10499 consensus_queue.cc:1035] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbf2942fb965445db54e6f1985e89499" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 46467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:34.043087 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:34.040366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:59501 (local address 127.7.250.66:37967)
0923 15:06:34.040408 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:34.040411 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:34.040545 (+ 134us) server_negotiation.cc:407] Connection header received
0923 15:06:34.040611 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:34.040612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:34.040623 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:34.040677 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:34.040782 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.041275 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.042339 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.042584 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.042697 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:34.042829 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:34.042861 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:34.042931 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:34.042984 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 15:06:34.043013 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:34.043108 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:34.040331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:37967 (local address 127.7.250.67:59501)
0923 15:06:34.040482 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:34.040500 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:34.040560 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:34.040681 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:34.040682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:34.040683 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:34.040735 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:34.040736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.041362 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.041363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:34.042250 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:34.042252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:34.042681 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:34.042683 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:34.042754 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:34.042758 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:34.042931 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:34.042933 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:34.042981 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:06:34.043030 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
W20260923 15:06:34.043532 10468 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 15:06:35.567549 9903 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:35.567662 9903 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:35.568576 9903 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26ac7307224d4e78b2765cc6bd5b807d: Sending DeleteTablet for 3 replicas of tablet b7cfca77f7214d23b9f473ef4ed11a34
I20260923 15:06:35.568840 10180 tablet_service.cc:1514] Processing DeleteTablet for tablet b7cfca77f7214d23b9f473ef4ed11a34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:35 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:34236
I20260923 15:06:35.568837 10037 tablet_service.cc:1514] Processing DeleteTablet for tablet b7cfca77f7214d23b9f473ef4ed11a34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:35 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32996
I20260923 15:06:35.568851 10319 tablet_service.cc:1514] Processing DeleteTablet for tablet b7cfca77f7214d23b9f473ef4ed11a34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:35 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37044
I20260923 15:06:35.569036 10551 tablet_replica.cc:330] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499: stopping tablet replica
I20260923 15:06:35.569110 10551 raft_consensus.cc:2229] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:35.569170 10551 raft_consensus.cc:2258] T b7cfca77f7214d23b9f473ef4ed11a34 P cbf2942fb965445db54e6f1985e89499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:35.569247 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9975
I20260923 15:06:35.569881 10550 tablet_replica.cc:330] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: stopping tablet replica
I20260923 15:06:35.569986 10550 raft_consensus.cc:2229] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:35.570093 10550 raft_consensus.cc:2258] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:35.570320 10550 ts_tablet_manager.cc:1905] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:35.571691 10552 tablet_replica.cc:330] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: stopping tablet replica
I20260923 15:06:35.571780 10552 raft_consensus.cc:2229] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:35.571827 10552 raft_consensus.cc:2258] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:35.571978 10550 ts_tablet_manager.cc:1918] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:06:35.572045 10550 log.cc:1196] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/b7cfca77f7214d23b9f473ef4ed11a34
I20260923 15:06:35.572290 10550 ts_tablet_manager.cc:1939] T b7cfca77f7214d23b9f473ef4ed11a34 P 5b04974649ad477ea37705e80acd3d1f: Deleting consensus metadata
I20260923 15:06:35.572600 9887 catalog_manager.cc:4841] TS 5b04974649ad477ea37705e80acd3d1f (127.7.250.66:37967): tablet b7cfca77f7214d23b9f473ef4ed11a34 (table test-table [id=823d32beff2b46689f79293ded474154]) successfully deleted
I20260923 15:06:35.574520 10552 ts_tablet_manager.cc:1905] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:35.576465 10552 ts_tablet_manager.cc:1918] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:06:35.576532 10552 log.cc:1196] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/b7cfca77f7214d23b9f473ef4ed11a34
W20260923 15:06:35.576668 9890 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:46467 (error 108)
I20260923 15:06:35.576792 10552 ts_tablet_manager.cc:1939] T b7cfca77f7214d23b9f473ef4ed11a34 P fb16d0bc51814f47bbf0d1f94f323002: Deleting consensus metadata
I20260923 15:06:35.576867 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 10115
I20260923 15:06:35.578194 9887 catalog_manager.cc:4841] TS fb16d0bc51814f47bbf0d1f94f323002 (127.7.250.67:42811): tablet b7cfca77f7214d23b9f473ef4ed11a34 (table test-table [id=823d32beff2b46689f79293ded474154]) successfully deleted
I20260923 15:06:35.578423 10554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:35.577797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:46467 (local address 127.0.0.1:60464)
0923 15:06:35.578326 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:35.578379 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:46467: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":63,"threads_started":1}
W20260923 15:06:35.578574 9890 catalog_manager.cc:4584] TS cbf2942fb965445db54e6f1985e89499 (127.7.250.65:46467): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b7cfca77f7214d23b9f473ef4ed11a34: Network error: Client connection negotiation failed: client connection to 127.7.250.65:46467: connect: Connection refused (error 111)
I20260923 15:06:35.583290 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 10255
I20260923 15:06:35.589671 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 9871
2026-09-23T15:06:35Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5725 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:35 dist-test-slave-bb3x krb5kdc[10557](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:35 dist-test-slave-bb3x krb5kdc[10557](info): set up 2 sockets
Sep 23 15:06:35 dist-test-slave-bb3x krb5kdc[10557](info): commencing operation
krb5kdc: starting...
W20260923 15:06:36.663833 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:36 dist-test-slave-bb3x krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:36.794502 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45983
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:45983
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:36.881294 10570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:36.881699 10570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:36.881763 10570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:36.883746 10570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:36.883854 10570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:36.883895 10570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:36.883932 10570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:36.883967 10570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:36.886351 10570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:45983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:36.886670 10570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:36.886992 10570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:36.892619 10578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:36.892619 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:36.892647 10575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:36.892822 10570 server_base.cc:1034] running on GCE node
I20260923 15:06:36.893100 10570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:36.893420 10570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:36.894609 10570 hybrid_clock.cc:648] HybridClock initialized: now 1790175996894555 us; error 62 us; skew 500 ppm
Sep 23 15:06:36 dist-test-slave-bb3x krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175996, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:36.897693 10570 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:36.898175 10570 webserver.cc:460] Webserver started at http://127.7.250.126:40621/ using document root <none> and password file <none>
I20260923 15:06:36.898463 10570 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:36.898540 10570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:36.898693 10570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:36.899869 10570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7"
format_stamp: "Formatted at 2026-09-23 15:06:36 on dist-test-slave-bb3x"
server_key: "96376439e73fc9abbf70bdf9610c1631"
server_key_iv: "348781955605ea78d5c7f04f037b26f0"
server_key_version: "encryptionkey@0"
I20260923 15:06:36.900254 10570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7"
format_stamp: "Formatted at 2026-09-23 15:06:36 on dist-test-slave-bb3x"
server_key: "96376439e73fc9abbf70bdf9610c1631"
server_key_iv: "348781955605ea78d5c7f04f037b26f0"
server_key_version: "encryptionkey@0"
I20260923 15:06:36.901749 10570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:36.902760 10585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:36.903023 10570 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:36.903167 10570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7"
format_stamp: "Formatted at 2026-09-23 15:06:36 on dist-test-slave-bb3x"
server_key: "96376439e73fc9abbf70bdf9610c1631"
server_key_iv: "348781955605ea78d5c7f04f037b26f0"
server_key_version: "encryptionkey@0"
I20260923 15:06:36.903252 10570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:36.930867 10570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:36.933511 10570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:36.933738 10570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:36.938056 10570 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:45983
I20260923 15:06:36.938089 10637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:45983 every 8 connection(s)
I20260923 15:06:36.938632 10570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:36.939769 10638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:36.939991 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 10570
I20260923 15:06:36.940066 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:36.940218 8169 external_mini_cluster.cc:1428] Setting key bc1d4e13cd15e381955a97d34b263c1b
I20260923 15:06:36.942732 10638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7: Bootstrap starting.
I20260923 15:06:36.943410 10638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:36.943715 10638 log.cc:824] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:36.944387 10638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7: No bootstrap required, opened a new log
I20260923 15:06:36.946408 10638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45983 } }
I20260923 15:06:36.946571 10638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:36.946635 10638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b7b84f15ca45bfa5dbb2dfbc6d4bd7, State: Initialized, Role: FOLLOWER
I20260923 15:06:36.946798 10638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45983 } }
I20260923 15:06:36.946897 10638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:36.946941 10638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:36.946991 10638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 0 FOLLOWER]: Advancing to term 1
Sep 23 15:06:36 dist-test-slave-bb3x krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175996, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:36.947707 10638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45983 } }
I20260923 15:06:36.947813 10638 leader_election.cc:302] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15b7b84f15ca45bfa5dbb2dfbc6d4bd7; no voters:
I20260923 15:06:36.948011 10638 leader_election.cc:288] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:36.948109 10643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:36.948261 10643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [term 1 LEADER]: Becoming Leader. State: Replica: 15b7b84f15ca45bfa5dbb2dfbc6d4bd7, State: Running, Role: LEADER
I20260923 15:06:36.948289 10638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:36.948357 10643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45983 } }
I20260923 15:06:36.948953 10645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15b7b84f15ca45bfa5dbb2dfbc6d4bd7. Latest consensus state: current_term: 1 leader_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45983 } } }
I20260923 15:06:36.949062 10645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:36.948933 10644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15b7b84f15ca45bfa5dbb2dfbc6d4bd7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45983 } } }
I20260923 15:06:36.949236 10644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:36.949354 10652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:36.949913 10652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:36.951586 10652 catalog_manager.cc:1347] Generated new cluster ID: e1ee0bbee28b4a7eb06cc9096dfc2b43
I20260923 15:06:36.951633 10652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:36.952667 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:36.940790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55734 (local address 127.7.250.126:45983)
0923 15:06:36.941010 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:36.941028 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:36.941064 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:06:36.941599 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:36.941608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:36.941667 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:36.941829 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:36.942593 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:36.943953 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:36.944792 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:36.945226 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:36.945345 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:36.948212 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:36.948217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:36.948217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:36.948232 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:36.951019 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:36.951440 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:36.951443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:36.951443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:36.951541 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:36.951757 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:36.951758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:36.951758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:36.951998 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:36.952194 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:36.952300 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 15:06:36.952441 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
I20260923 15:06:36.970340 10652 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:36.970914 10652 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:36.974458 10652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15b7b84f15ca45bfa5dbb2dfbc6d4bd7: Generated new TSK 0
I20260923 15:06:36.974707 10652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:37.009816 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45983
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:37.094102 10664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:37.094450 10664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:37.094497 10664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:37.096561 10664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:37.096637 10664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:37.096684 10664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:37.099184 10664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:37.099406 10664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:37.099613 10664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:37.104905 10672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:37.104908 10670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:37.104879 10669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:37.105134 10664 server_base.cc:1034] running on GCE node
I20260923 15:06:37.105348 10664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:37.105636 10664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:37.106809 10664 hybrid_clock.cc:648] HybridClock initialized: now 1790175997106770 us; error 63 us; skew 500 ppm
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:37.109632 10664 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:37.110116 10664 webserver.cc:460] Webserver started at http://127.7.250.65:40791/ using document root <none> and password file <none>
I20260923 15:06:37.110433 10664 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:37.110486 10664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:37.110630 10664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:37.111716 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "622c3e61099640cab5e3c075fb30eb7a"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "d426906438695b0bdf864054732bbf13"
server_key_iv: "36b93204ac9c209b8089545f72d438e2"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.112097 10664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "622c3e61099640cab5e3c075fb30eb7a"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "d426906438695b0bdf864054732bbf13"
server_key_iv: "36b93204ac9c209b8089545f72d438e2"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.113456 10664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:37.114454 10679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.114744 10664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:37.114828 10664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "622c3e61099640cab5e3c075fb30eb7a"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "d426906438695b0bdf864054732bbf13"
server_key_iv: "36b93204ac9c209b8089545f72d438e2"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.114923 10664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:37.150640 10664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:37.152643 10664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:37.152760 10664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:37.152953 10664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:37.153298 10664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:37.153332 10664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.153355 10664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:37.153406 10664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.159328 10664 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:38689
I20260923 15:06:37.159371 10792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:38689 every 8 connection(s)
I20260923 15:06:37.159731 10664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:37.165974 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 10664
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:37.166087 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:37.166352 8169 external_mini_cluster.cc:1428] Setting key fe0cba4e12437121f5ac6a7e59019539
I20260923 15:06:37.170102 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:37.160517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45983 (local address 127.7.250.65:36483)
0923 15:06:37.160691 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:37.160735 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:37.161272 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:37.161530 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:37.161534 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:37.162155 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:37.162381 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:37.162386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.163130 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.163131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:37.163917 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:37.163919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.164328 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.164329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:37.164453 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:37.164962 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:37.164970 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:37.166600 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:37.168682 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:37.168689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:37.168691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:37.168920 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:37.169136 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:37.169136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:37.169137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:37.169236 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:37.169566 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:37.169573 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:37.169701 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 15:06:37.169800 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 15:06:37.169933 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260923 15:06:37.170089 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:37.160544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:36483 (local address 127.7.250.126:45983)
0923 15:06:37.160629 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:37.160631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:37.160825 (+ 194us) server_negotiation.cc:407] Connection header received
0923 15:06:37.161390 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:37.161391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:37.161404 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:37.161460 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:37.162493 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.163017 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.164020 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.164227 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.164317 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:37.166704 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:37.166706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:37.166706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:37.166714 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:37.168584 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:37.168998 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:37.168999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:37.168999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:37.169069 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:37.169305 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:37.169307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:37.169307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:37.169457 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:37.169524 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:37.169840 (+ 316us) server_negotiation.cc:299] Negotiation successful
0923 15:06:37.169943 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:06:37.170866 10793 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45983
I20260923 15:06:37.170984 10793 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:37.171214 10793 heartbeater.cc:511] Master 127.7.250.126:45983 requested a full tablet report, sending...
I20260923 15:06:37.171759 10602 ts_manager.cc:194] Registered new tserver with Master: 622c3e61099640cab5e3c075fb30eb7a (127.7.250.65:38689)
I20260923 15:06:37.172415 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:36483
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:37.194550 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45983
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:37.277555 10798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:37.277832 10798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:37.277873 10798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:37.279170 10798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:37.279254 10798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:37.279304 10798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:37.280829 10798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:37.281122 10798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:37.281368 10798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:37.286628 10803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:37.286589 10806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:37.286696 10804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:37.286801 10798 server_base.cc:1034] running on GCE node
I20260923 15:06:37.287134 10798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:37.287397 10798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:37.288563 10798 hybrid_clock.cc:648] HybridClock initialized: now 1790175997288557 us; error 36 us; skew 500 ppm
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:37.291282 10798 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:37.291790 10798 webserver.cc:460] Webserver started at http://127.7.250.66:44639/ using document root <none> and password file <none>
I20260923 15:06:37.292037 10798 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:37.292109 10798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:37.292256 10798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:37.293358 10798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "ee673cae324c4d5fa8fe83a5854b877c"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "5600e7b35489f703bc702466c10a3d78"
server_key_iv: "5ee75784bc3069bcb07f8ada0457989a"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.293761 10798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "ee673cae324c4d5fa8fe83a5854b877c"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "5600e7b35489f703bc702466c10a3d78"
server_key_iv: "5ee75784bc3069bcb07f8ada0457989a"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.295244 10798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:37.296088 10813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.296259 10798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.296370 10798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "ee673cae324c4d5fa8fe83a5854b877c"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "5600e7b35489f703bc702466c10a3d78"
server_key_iv: "5ee75784bc3069bcb07f8ada0457989a"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.296454 10798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:37.312754 10798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:37.315177 10798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:37.315347 10798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:37.315608 10798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:37.315933 10798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:37.315989 10798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.316054 10798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:37.316085 10798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.322398 10798 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:36741
I20260923 15:06:37.322438 10926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:36741 every 8 connection(s)
I20260923 15:06:37.322768 10798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:37.329612 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 10798
I20260923 15:06:37.329723 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:37.329944 8169 external_mini_cluster.cc:1428] Setting key 7c2acd997ea3dd29965a0e4ceb201752
I20260923 15:06:37.332969 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:37.323553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:52885 (local address 127.7.250.126:45983)
0923 15:06:37.323604 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:37.323606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:37.323953 (+ 347us) server_negotiation.cc:407] Connection header received
0923 15:06:37.324507 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:37.324508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:37.324520 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:37.324589 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:37.325610 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.326139 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.327202 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.327408 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.327496 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:37.330038 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:37.330040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:37.330040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:37.330048 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:37.331535 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:37.331972 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:37.331974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:37.331974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:37.332035 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:37.332292 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:37.332293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:37.332293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:37.332412 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:37.332495 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:37.332779 (+ 284us) server_negotiation.cc:299] Negotiation successful
0923 15:06:37.332868 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:37.333154 10929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:37.323646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45983 (local address 127.7.250.66:52885)
0923 15:06:37.323824 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:37.323864 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:37.324392 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:37.324675 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:37.324679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:37.325279 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:37.325486 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:37.325490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.326243 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.326244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:37.327097 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:37.327100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.327510 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.327511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:37.327641 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:37.328332 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:37.328341 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:37.329935 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:37.331645 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:37.331653 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:37.331655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:37.331884 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:37.332107 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:37.332108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:37.332108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:37.332224 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:37.332484 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:37.332491 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:37.332638 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 15:06:37.332745 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 15:06:37.332911 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260923 15:06:37.333642 10927 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45983
I20260923 15:06:37.333765 10927 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:37.333951 10927 heartbeater.cc:511] Master 127.7.250.126:45983 requested a full tablet report, sending...
I20260923 15:06:37.334429 10602 ts_manager.cc:194] Registered new tserver with Master: ee673cae324c4d5fa8fe83a5854b877c (127.7.250.66:36741)
I20260923 15:06:37.334991 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:52885
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:37.361423 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45983
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:37.443965 10932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:37.444226 10932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:37.444263 10932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:37.445434 10932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:37.445518 10932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:37.445565 10932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:37.447211 10932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45983
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:37.447510 10932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:37.447743 10932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:37.453334 10932 server_base.cc:1034] running on GCE node
W20260923 15:06:37.453263 10940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:37.453263 10937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:37.453279 10938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:37.453739 10932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:37.453995 10932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:37.455173 10932 hybrid_clock.cc:648] HybridClock initialized: now 1790175997455146 us; error 52 us; skew 500 ppm
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[10557](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:37.457685 10932 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:37.458159 10932 webserver.cc:460] Webserver started at http://127.7.250.67:32839/ using document root <none> and password file <none>
I20260923 15:06:37.458427 10932 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:37.458475 10932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:37.458570 10932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:37.459578 10932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "10088280d8f74c269d84cbacad593c4e"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "60561ff8dd0e01369d1ee00fd369f2b1"
server_key_iv: "d3df283e5738829b86c2b2aba319acc3"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.459889 10932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "10088280d8f74c269d84cbacad593c4e"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "60561ff8dd0e01369d1ee00fd369f2b1"
server_key_iv: "d3df283e5738829b86c2b2aba319acc3"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.461259 10932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:37.462037 10947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.462245 10932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:37.462352 10932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "10088280d8f74c269d84cbacad593c4e"
format_stamp: "Formatted at 2026-09-23 15:06:37 on dist-test-slave-bb3x"
server_key: "60561ff8dd0e01369d1ee00fd369f2b1"
server_key_iv: "d3df283e5738829b86c2b2aba319acc3"
server_key_version: "encryptionkey@0"
I20260923 15:06:37.462419 10932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:37.472744 10932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:37.475162 10932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:37.475272 10932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:37.475438 10932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:37.475699 10932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:37.475731 10932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.475754 10932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:37.475766 10932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:37.481527 10932 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:34369
I20260923 15:06:37.481606 11060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:34369 every 8 connection(s)
I20260923 15:06:37.481922 10932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:37.486456 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 10932
I20260923 15:06:37.486598 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:37.486799 8169 external_mini_cluster.cc:1428] Setting key 4a7c35d2f7242b1cb734ca25f943d89b
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[10557](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175997, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:37.492182 10641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:37.482726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:52069 (local address 127.7.250.126:45983)
0923 15:06:37.482805 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:37.482808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:37.482985 (+ 177us) server_negotiation.cc:407] Connection header received
0923 15:06:37.483502 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:37.483503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:37.483516 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:37.483561 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:37.484728 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.485263 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.486339 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.486617 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.486690 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:37.489250 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:37.489251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:37.489251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:37.489259 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:37.490748 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:37.491185 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:37.491186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:37.491187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:37.491246 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:37.491503 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:37.491504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:37.491505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:37.491643 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:37.491725 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:37.491995 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 15:06:37.492092 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:06:37.492380 11063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:37.482692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45983 (local address 127.7.250.67:52069)
0923 15:06:37.482863 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:37.482898 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:37.483395 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:37.483713 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:37.483717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:37.484322 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:37.484595 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:37.484600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.485382 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.485383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:37.486195 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:37.486197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:37.486714 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:37.486716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:37.486849 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:37.487500 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:37.487508 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:37.489122 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:37.490846 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:37.490854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:37.490856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:37.491092 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:37.491321 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:37.491322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:37.491322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:37.491434 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:37.491718 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:37.491725 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:37.491865 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 15:06:37.491975 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 15:06:37.492143 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260923 15:06:37.492982 11061 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45983
I20260923 15:06:37.493074 11061 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:37.493225 11061 heartbeater.cc:511] Master 127.7.250.126:45983 requested a full tablet report, sending...
I20260923 15:06:37.493562 10602 ts_manager.cc:194] Registered new tserver with Master: 10088280d8f74c269d84cbacad593c4e (127.7.250.67:34369)
I20260923 15:06:37.493930 10602 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:52069
I20260923 15:06:37.498929 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 15:06:37.511025 10641 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:37.509829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55748 (local address 127.7.250.126:45983)
0923 15:06:37.509908 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:37.509910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:37.510066 (+ 156us) server_negotiation.cc:407] Connection header received
0923 15:06:37.510096 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:37.510097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:37.510108 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:37.510168 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:37.510898 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55748: BlockingRecv error: recv got EOF from 127.0.0.1:55748 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 15:06:37.511781 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 10664
I20260923 15:06:37.516428 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 10798
I20260923 15:06:37.521651 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 10932
I20260923 15:06:37.526212 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 10570
2026-09-23T15:06:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1935 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[11073](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[11073](info): set up 2 sockets
Sep 23 15:06:37 dist-test-slave-bb3x krb5kdc[11073](info): commencing operation
krb5kdc: starting...
W20260923 15:06:38.607813 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:38 dist-test-slave-bb3x krb5kdc[11073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:38.733764 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45323
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:45323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:38.824247 11086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:38.824465 11086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:38.824489 11086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:38.825729 11086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:38.825778 11086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:38.825793 11086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:38.825805 11086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:38.825827 11086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:38.827399 11086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:45323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:38.827605 11086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:38.827776 11086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:38.832558 11091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:38.832672 11092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:38.832698 11094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:38.832767 11086 server_base.cc:1034] running on GCE node
I20260923 15:06:38.833081 11086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:38.833319 11086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:38.834443 11086 hybrid_clock.cc:648] HybridClock initialized: now 1790175998834422 us; error 34 us; skew 500 ppm
Sep 23 15:06:38 dist-test-slave-bb3x krb5kdc[11073](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175998, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:38.836793 11086 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:38.837255 11086 webserver.cc:460] Webserver started at http://127.7.250.126:41199/ using document root <none> and password file <none>
I20260923 15:06:38.837497 11086 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:38.837548 11086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:38.837683 11086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:38.838840 11086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "8f10ad6522d34a35b80a5152b74e65fb"
format_stamp: "Formatted at 2026-09-23 15:06:38 on dist-test-slave-bb3x"
server_key: "627594974fdac37467c90a09c1bfb158"
server_key_iv: "4e0ada59ba02cdd7059007762fda9288"
server_key_version: "encryptionkey@0"
I20260923 15:06:38.839216 11086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "8f10ad6522d34a35b80a5152b74e65fb"
format_stamp: "Formatted at 2026-09-23 15:06:38 on dist-test-slave-bb3x"
server_key: "627594974fdac37467c90a09c1bfb158"
server_key_iv: "4e0ada59ba02cdd7059007762fda9288"
server_key_version: "encryptionkey@0"
I20260923 15:06:38.840644 11086 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:38.841441 11101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:38.841658 11086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:38.841765 11086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "8f10ad6522d34a35b80a5152b74e65fb"
format_stamp: "Formatted at 2026-09-23 15:06:38 on dist-test-slave-bb3x"
server_key: "627594974fdac37467c90a09c1bfb158"
server_key_iv: "4e0ada59ba02cdd7059007762fda9288"
server_key_version: "encryptionkey@0"
I20260923 15:06:38.841850 11086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:38.854135 11086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:38.856199 11086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:38.856376 11086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:38.860594 11086 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:45323
I20260923 15:06:38.860633 11153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:45323 every 8 connection(s)
I20260923 15:06:38.861034 11086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:38.861969 11154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:38.864307 11154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb: Bootstrap starting.
I20260923 15:06:38.864864 11154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:38.865114 11154 log.cc:824] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:38.865720 11154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb: No bootstrap required, opened a new log
I20260923 15:06:38.867731 11154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f10ad6522d34a35b80a5152b74e65fb" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45323 } }
I20260923 15:06:38.867873 11154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:38.867895 11154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f10ad6522d34a35b80a5152b74e65fb, State: Initialized, Role: FOLLOWER
I20260923 15:06:38.867998 11154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f10ad6522d34a35b80a5152b74e65fb" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45323 } }
I20260923 15:06:38.868057 11154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:38.868083 11154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:38.868112 11154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:38.868937 11154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f10ad6522d34a35b80a5152b74e65fb" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45323 } }
I20260923 15:06:38.869038 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11086
I20260923 15:06:38.869072 11154 leader_election.cc:302] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f10ad6522d34a35b80a5152b74e65fb; no voters:
I20260923 15:06:38.869112 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:38.869253 11154 leader_election.cc:288] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:38.869299 8169 external_mini_cluster.cc:1428] Setting key 485fbebd65f0e95e4de32023eb959b72
I20260923 15:06:38.869333 11157 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:38.869509 11157 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [term 1 LEADER]: Becoming Leader. State: Replica: 8f10ad6522d34a35b80a5152b74e65fb, State: Running, Role: LEADER
I20260923 15:06:38.869535 11154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:38.869611 11157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f10ad6522d34a35b80a5152b74e65fb" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45323 } }
I20260923 15:06:38.869921 11157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f10ad6522d34a35b80a5152b74e65fb. Latest consensus state: current_term: 1 leader_uuid: "8f10ad6522d34a35b80a5152b74e65fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f10ad6522d34a35b80a5152b74e65fb" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45323 } } }
I20260923 15:06:38.869995 11157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:38.870136 11158 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f10ad6522d34a35b80a5152b74e65fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f10ad6522d34a35b80a5152b74e65fb" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45323 } } }
I20260923 15:06:38.870275 11158 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:38.870829 11164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:38.871346 11164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:38.873802 11164 catalog_manager.cc:1347] Generated new cluster ID: 5ad12599912f4af99f74737496d7ebb1
I20260923 15:06:38.873867 11164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:38 dist-test-slave-bb3x krb5kdc[11073](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:38.880518 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:38.869900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50174 (local address 127.7.250.126:45323)
0923 15:06:38.870302 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:38.870312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:38.870350 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:06:38.870473 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:38.870480 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:38.870608 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:38.871344 (+ 736us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:38.871766 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:38.873238 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:38.874045 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:38.874541 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:38.874697 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:38.877053 (+ 2356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:38.877057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:38.877057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:38.877069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:38.878867 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:38.879263 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:38.879266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:38.879266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:38.879369 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:38.879588 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:38.879589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:38.879589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:38.879843 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:38.880030 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:38.880098 (+ 68us) server_negotiation.cc:299] Negotiation successful
0923 15:06:38.880345 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":162,"threads_started":1}
I20260923 15:06:38.881393 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:38.885850 11164 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:38.886608 11164 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:38.895164 11164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f10ad6522d34a35b80a5152b74e65fb: Generated new TSK 0
I20260923 15:06:38.895371 11164 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:38.899864 11162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:38.895530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50180 (local address 127.7.250.126:45323)
0923 15:06:38.895587 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:38.895589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:38.895915 (+ 326us) server_negotiation.cc:407] Connection header received
0923 15:06:38.895994 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:38.895995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:38.896021 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:38.896072 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:38.896620 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:38.897412 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:38.898187 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:38.898492 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:38.898591 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:38.899325 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:38.899327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:38.899327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:38.899523 (+ 196us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:38.899540 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:38.899547 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:38.899688 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:38.899748 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 15:06:38.899774 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:38.902667 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11086
2026-09-23T15:06:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1373 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:38 dist-test-slave-bb3x krb5kdc[11187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:38 dist-test-slave-bb3x krb5kdc[11187](info): set up 2 sockets
Sep 23 15:06:38 dist-test-slave-bb3x krb5kdc[11187](info): commencing operation
krb5kdc: starting...
W20260923 15:06:39.996183 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:40.126557 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41229
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:40.208860 11200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:40.209153 11200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:40.209197 11200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:40.210465 11200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:40.210544 11200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:40.210572 11200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:40.210603 11200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:40.210623 11200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:40.212132 11200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:40.212417 11200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:40.212661 11200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:40.217871 11208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.217861 11205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.217896 11206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:40.218051 11200 server_base.cc:1034] running on GCE node
I20260923 15:06:40.218477 11200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:40.218809 11200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:40.219955 11200 hybrid_clock.cc:648] HybridClock initialized: now 1790176000219934 us; error 35 us; skew 500 ppm
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:40.222530 11200 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:40.222970 11200 webserver.cc:460] Webserver started at http://127.7.250.126:43611/ using document root <none> and password file <none>
I20260923 15:06:40.223201 11200 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:40.223272 11200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:40.223416 11200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:40.224452 11200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "4c069b773ca04bc8a01a98b7f00cdba5"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "00ed3e63c25702405a62414ead105678"
server_key_iv: "0bcf9ed8b04dd78a56c352a40f285a92"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.224817 11200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "4c069b773ca04bc8a01a98b7f00cdba5"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "00ed3e63c25702405a62414ead105678"
server_key_iv: "0bcf9ed8b04dd78a56c352a40f285a92"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.226152 11200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:40.226984 11215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.227190 11200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:40.227288 11200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "4c069b773ca04bc8a01a98b7f00cdba5"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "00ed3e63c25702405a62414ead105678"
server_key_iv: "0bcf9ed8b04dd78a56c352a40f285a92"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.227371 11200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:40.243355 11200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:40.245730 11200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:40.245898 11200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:40.250432 11200 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41229
I20260923 15:06:40.250487 11267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41229 every 8 connection(s)
I20260923 15:06:40.250860 11200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:40.252012 11268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:40.252085 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11200
I20260923 15:06:40.252219 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:40.252398 8169 external_mini_cluster.cc:1428] Setting key 2ac71449e87d286a70486b64873a7c52
I20260923 15:06:40.255429 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5: Bootstrap starting.
I20260923 15:06:40.256197 11268 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:40.256518 11268 log.cc:824] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:40.257292 11268 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5: No bootstrap required, opened a new log
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:40.259779 11268 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41229 } }
I20260923 15:06:40.259917 11268 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:40.259939 11268 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c069b773ca04bc8a01a98b7f00cdba5, State: Initialized, Role: FOLLOWER
I20260923 15:06:40.260082 11268 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41229 } }
I20260923 15:06:40.260188 11268 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:40.260224 11268 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:40.260269 11268 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:40.260871 11268 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41229 } }
I20260923 15:06:40.260982 11268 leader_election.cc:302] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c069b773ca04bc8a01a98b7f00cdba5; no voters:
I20260923 15:06:40.261155 11268 leader_election.cc:288] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:40.261185 11273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:40.261291 11273 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [term 1 LEADER]: Becoming Leader. State: Replica: 4c069b773ca04bc8a01a98b7f00cdba5, State: Running, Role: LEADER
I20260923 15:06:40.261381 11273 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41229 } }
I20260923 15:06:40.261442 11268 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:40.261685 11273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c069b773ca04bc8a01a98b7f00cdba5. Latest consensus state: current_term: 1 leader_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41229 } } }
I20260923 15:06:40.261765 11273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:40.261698 11274 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c069b773ca04bc8a01a98b7f00cdba5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41229 } } }
I20260923 15:06:40.261834 11274 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:40.263188 11288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:40.263837 11288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:40.264118 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.253004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.7.250.126:41229)
0923 15:06:40.253186 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:40.253194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:40.253220 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:06:40.253640 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:40.253646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:40.253692 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:40.253805 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:40.254645 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.255511 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.256627 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.257042 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.257125 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.259775 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:40.259784 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:40.259784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:40.259795 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:40.262169 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.262682 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.262686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.262686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.262852 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.263114 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.263115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.263115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.263442 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:40.263569 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:40.263762 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 15:06:40.263917 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260923 15:06:40.265563 11288 catalog_manager.cc:1347] Generated new cluster ID: 7a348472e6e0495c819add2cee4ae74f
I20260923 15:06:40.265614 11288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:40.282037 11288 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:40.282694 11288 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:40.288919 11288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c069b773ca04bc8a01a98b7f00cdba5: Generated new TSK 0
I20260923 15:06:40.289177 11288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:40.321746 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41229
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:40.406438 11294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:40.406666 11294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:40.406692 11294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:40.407863 11294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:40.407907 11294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:40.407932 11294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:40.409351 11294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:40.409565 11294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:40.409760 11294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:40.415130 11299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:40.415156 11294 server_base.cc:1034] running on GCE node
W20260923 15:06:40.415138 11300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.415138 11302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:40.415684 11294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:40.415951 11294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:40.417186 11294 hybrid_clock.cc:648] HybridClock initialized: now 1790176000417147 us; error 53 us; skew 500 ppm
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:40.420024 11294 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:40.420480 11294 webserver.cc:460] Webserver started at http://127.7.250.65:43693/ using document root <none> and password file <none>
I20260923 15:06:40.420714 11294 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:40.420784 11294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:40.420934 11294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:40.422009 11294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "1583a5940cf444fba4663a901f4457b7"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "0aa71acbd6d6146de996db1cea854dcd"
server_key_iv: "a512bb96792cb7644f7bd84e4fd297b5"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.422410 11294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "1583a5940cf444fba4663a901f4457b7"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "0aa71acbd6d6146de996db1cea854dcd"
server_key_iv: "a512bb96792cb7644f7bd84e4fd297b5"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.423801 11294 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:40.424638 11309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.424856 11294 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:40.424981 11294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "1583a5940cf444fba4663a901f4457b7"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "0aa71acbd6d6146de996db1cea854dcd"
server_key_iv: "a512bb96792cb7644f7bd84e4fd297b5"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.425060 11294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:40.433709 11294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:40.435737 11294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:40.435911 11294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:40.436161 11294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:40.436484 11294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:40.436539 11294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.436574 11294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:40.436601 11294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.442507 11294 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:41633
I20260923 15:06:40.442580 11422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:41633 every 8 connection(s)
I20260923 15:06:40.442919 11294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:40.446981 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11294
I20260923 15:06:40.447098 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:40.447310 8169 external_mini_cluster.cc:1428] Setting key 208d30e1fcfc3e47c3bcf136c0af67e7
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:40.453749 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.443709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:34603 (local address 127.7.250.126:41229)
0923 15:06:40.443760 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:40.443762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:40.444030 (+ 268us) server_negotiation.cc:407] Connection header received
0923 15:06:40.444637 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:40.444638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:40.444648 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:40.444714 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:40.445770 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.446354 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.447833 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.448103 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.448180 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.450619 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:40.450622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:40.450623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:40.450631 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:40.452192 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.452658 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.452659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.452660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.452726 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.453006 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.453007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.453008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.453138 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:40.453247 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:40.453552 (+ 305us) server_negotiation.cc:299] Negotiation successful
0923 15:06:40.453647 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:06:40.454021 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.443661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41229 (local address 127.7.250.65:34603)
0923 15:06:40.443884 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:40.443930 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:40.444442 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:40.444792 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:40.444796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:40.445408 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:40.445660 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:40.445665 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.446439 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.446440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:40.447748 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:40.447751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.448188 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.448189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:40.448320 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.448900 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:40.448909 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:40.450507 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:40.452305 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:40.452321 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:40.452323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:40.452571 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:40.452805 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:40.452806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:40.452806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:40.452930 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:40.453213 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:40.453220 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:40.453389 (+ 169us) client_negotiation.cc:769] Sending connection context
0923 15:06:40.453528 (+ 139us) client_negotiation.cc:240] Negotiation successful
0923 15:06:40.453705 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":113,"threads_started":1}
I20260923 15:06:40.454582 11423 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41229
I20260923 15:06:40.454703 11423 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:40.454928 11423 heartbeater.cc:511] Master 127.7.250.126:41229 requested a full tablet report, sending...
I20260923 15:06:40.455488 11232 ts_manager.cc:194] Registered new tserver with Master: 1583a5940cf444fba4663a901f4457b7 (127.7.250.65:41633)
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
I20260923 15:06:40.456261 11232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:34603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:40.478425 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41229
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:40.559568 11428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:40.559792 11428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:40.559816 11428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:40.560990 11428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:40.561061 11428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:40.561108 11428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:40.562885 11428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:40.563182 11428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:40.563421 11428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:40.569088 11436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.569082 11434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.569082 11433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:40.569123 11428 server_base.cc:1034] running on GCE node
I20260923 15:06:40.569525 11428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:40.569806 11428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:40.570999 11428 hybrid_clock.cc:648] HybridClock initialized: now 1790176000570982 us; error 27 us; skew 500 ppm
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:40.573745 11428 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:40.574182 11428 webserver.cc:460] Webserver started at http://127.7.250.66:40357/ using document root <none> and password file <none>
I20260923 15:06:40.574456 11428 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:40.574504 11428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:40.574635 11428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:40.575830 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "45870abeb6884182a10c08692799f626"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "98abc094248452799b9c7f89ba7a8a6d"
server_key_iv: "9ba4f7597471f989493a8ccea8504504"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.576192 11428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "45870abeb6884182a10c08692799f626"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "98abc094248452799b9c7f89ba7a8a6d"
server_key_iv: "9ba4f7597471f989493a8ccea8504504"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.577663 11428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:40.578605 11443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.578891 11428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:40.579020 11428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "45870abeb6884182a10c08692799f626"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "98abc094248452799b9c7f89ba7a8a6d"
server_key_iv: "9ba4f7597471f989493a8ccea8504504"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.579104 11428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:40.604714 11428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:40.606984 11428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:40.607151 11428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:40.607446 11428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:40.607813 11428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:40.607877 11428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.607913 11428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:40.607933 11428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.614969 11428 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:36535
I20260923 15:06:40.615033 11556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:36535 every 8 connection(s)
I20260923 15:06:40.615366 11428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:40.624006 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11428
I20260923 15:06:40.624135 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:40.624368 8169 external_mini_cluster.cc:1428] Setting key b281eabe0eae7853b1b655a39050a047
I20260923 15:06:40.625733 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.616155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41229 (local address 127.7.250.66:51465)
0923 15:06:40.616348 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:40.616391 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:40.616920 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:40.617180 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:40.617184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:40.617799 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:40.617998 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:40.618003 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.618860 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.618861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:40.619699 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:40.619702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.620076 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.620076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:40.620181 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.620682 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:40.620690 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:40.622071 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:40.623942 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:40.623950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:40.623953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:40.624333 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:40.624648 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:40.624650 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:40.624650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:40.624819 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:40.625144 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:40.625152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:40.625314 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 15:06:40.625409 (+ 95us) client_negotiation.cc:240] Negotiation successful
0923 15:06:40.625551 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":120,"threads_started":1}
I20260923 15:06:40.625764 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.616213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:51465 (local address 127.7.250.126:41229)
0923 15:06:40.616285 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:40.616287 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:40.616568 (+ 281us) server_negotiation.cc:407] Connection header received
0923 15:06:40.617027 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:40.617028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:40.617041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:40.617113 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:40.618098 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.618770 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.619791 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.619989 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.620053 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.622163 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:40.622164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:40.622165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:40.622172 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:40.623835 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.624448 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.624449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.624450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.624548 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.624901 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.624902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.624903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.625066 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:40.625141 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:40.625518 (+ 377us) server_negotiation.cc:299] Negotiation successful
0923 15:06:40.625613 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:06:40.626859 11557 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41229
I20260923 15:06:40.626963 11557 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:40.627167 11557 heartbeater.cc:511] Master 127.7.250.126:41229 requested a full tablet report, sending...
I20260923 15:06:40.627610 11232 ts_manager.cc:194] Registered new tserver with Master: 45870abeb6884182a10c08692799f626 (127.7.250.66:36535)
I20260923 15:06:40.628171 11232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:51465
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:40.654506 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41229
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:40.735764 11562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:40.736023 11562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:40.736063 11562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:40.737224 11562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:40.737296 11562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:40.737340 11562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:40.738893 11562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:40.739161 11562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:40.739404 11562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:40.744614 11568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.744630 11570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:40.744764 11567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:40.744817 11562 server_base.cc:1034] running on GCE node
I20260923 15:06:40.745038 11562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:40.745288 11562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:40.746471 11562 hybrid_clock.cc:648] HybridClock initialized: now 1790176000746450 us; error 28 us; skew 500 ppm
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:40.752511 11562 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:40.752978 11562 webserver.cc:460] Webserver started at http://127.7.250.67:37253/ using document root <none> and password file <none>
I20260923 15:06:40.753213 11562 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:40.753259 11562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:40.753402 11562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:40.754603 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "53dd33ecd40141ef82d380c570332198"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "9c37cd5f0ecc3eac706eaef404a1d8fe"
server_key_iv: "f3c9ee9efd17199d25922e0e8d25f632"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.754961 11562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "53dd33ecd40141ef82d380c570332198"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "9c37cd5f0ecc3eac706eaef404a1d8fe"
server_key_iv: "f3c9ee9efd17199d25922e0e8d25f632"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.756316 11562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:40.757258 11577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.757591 11562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:40.757673 11562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "53dd33ecd40141ef82d380c570332198"
format_stamp: "Formatted at 2026-09-23 15:06:40 on dist-test-slave-bb3x"
server_key: "9c37cd5f0ecc3eac706eaef404a1d8fe"
server_key_iv: "f3c9ee9efd17199d25922e0e8d25f632"
server_key_version: "encryptionkey@0"
I20260923 15:06:40.757783 11562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:40.771102 11562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:40.773186 11562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:40.773346 11562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:40.773630 11562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:40.773972 11562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:40.774004 11562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.774027 11562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:40.774075 11562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:40.780470 11562 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:45165
I20260923 15:06:40.780515 11690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:45165 every 8 connection(s)
I20260923 15:06:40.780860 11562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:40.789727 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11562
I20260923 15:06:40.789813 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:40.790016 8169 external_mini_cluster.cc:1428] Setting key b61de77524e614865a4484de2e8bf2d4
I20260923 15:06:40.791548 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.781778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:36063 (local address 127.7.250.126:41229)
0923 15:06:40.781863 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:40.781865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:40.782010 (+ 145us) server_negotiation.cc:407] Connection header received
0923 15:06:40.782646 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:40.782648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:40.782658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:40.782713 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:40.783891 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.784388 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.785520 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.785702 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.785760 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.788132 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:40.788133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:40.788134 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:40.788141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:40.789859 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.790396 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.790397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.790398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.790483 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.790804 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.790805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.790805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.790944 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:40.791010 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:40.791369 (+ 359us) server_negotiation.cc:299] Negotiation successful
0923 15:06:40.791434 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 15:06:40.791659 11693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.781689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41229 (local address 127.7.250.67:36063)
0923 15:06:40.781837 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:40.781927 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:40.782550 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:40.782770 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:40.782774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:40.783584 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:40.783795 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:40.783800 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.784502 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.784503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:40.785424 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:40.785427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.785811 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.785812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:40.785928 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.786546 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:40.786555 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:40.788036 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:40.789966 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:40.789973 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:40.789976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:40.790323 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:40.790557 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:40.790558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:40.790558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:40.790714 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:40.791017 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:40.791024 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:40.791241 (+ 217us) client_negotiation.cc:769] Sending connection context
0923 15:06:40.791327 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:06:40.791461 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260923 15:06:40.792114 11691 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41229
I20260923 15:06:40.792225 11691 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:40.792404 11691 heartbeater.cc:511] Master 127.7.250.126:41229 requested a full tablet report, sending...
I20260923 15:06:40.792773 11232 ts_manager.cc:194] Registered new tserver with Master: 53dd33ecd40141ef82d380c570332198 (127.7.250.67:45165)
I20260923 15:06:40.793108 11232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:36063
I20260923 15:06:40.802067 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176000, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:40.825270 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:40.817788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41426 (local address 127.7.250.126:41229)
0923 15:06:40.817875 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:40.817877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:40.817985 (+ 108us) server_negotiation.cc:407] Connection header received
0923 15:06:40.818060 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:40.818060 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:40.818071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:40.818151 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:40.818848 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.819340 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.820084 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:40.820299 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:40.820389 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:40.822666 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:40.822667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:40.822668 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:40.822676 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:40.824080 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.824451 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.824452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.824453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.824513 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:40.824714 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:40.824715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:40.824716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:40.824871 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:40.824985 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:40.825087 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 15:06:40.825165 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260923 15:06:40.825577 11232 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41426
I20260923 15:06:40.826076 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11294
I20260923 15:06:40.831146 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11428
I20260923 15:06:40.835839 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11562
I20260923 15:06:40.840121 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11200
2026-09-23T15:06:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1939 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11703](info): set up 2 sockets
Sep 23 15:06:40 dist-test-slave-bb3x krb5kdc[11703](info): commencing operation
krb5kdc: starting...
W20260923 15:06:41.908808 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:41 dist-test-slave-bb3x krb5kdc[11703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:42.032852 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44751
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:44751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:42.114472 11716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:42.114693 11716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:42.114718 11716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:42.115885 11716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:42.115918 11716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:42.115932 11716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:42.115944 11716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:42.115957 11716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:42.117421 11716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:44751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:42.117631 11716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:42.117805 11716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:42.123073 11721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:42.123073 11724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.123103 11716 server_base.cc:1034] running on GCE node
W20260923 15:06:42.123145 11722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.123643 11716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:42.123997 11716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:42.125207 11716 hybrid_clock.cc:648] HybridClock initialized: now 1790176002125193 us; error 55 us; skew 500 ppm
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:42.128269 11716 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:42.128875 11716 webserver.cc:460] Webserver started at http://127.7.250.126:36981/ using document root <none> and password file <none>
I20260923 15:06:42.129122 11716 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:42.129195 11716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:42.129357 11716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:42.130548 11716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "eeda85a3b21d493ca6bf9a8b3d034128"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "a3d4daadaf8ef02289cacc0f2740e3f9"
server_key_iv: "dfcb6f563eeb139fe89016f980c823b1"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.130919 11716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "eeda85a3b21d493ca6bf9a8b3d034128"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "a3d4daadaf8ef02289cacc0f2740e3f9"
server_key_iv: "dfcb6f563eeb139fe89016f980c823b1"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.132306 11716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:42.133170 11731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.133353 11716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:42.133513 11716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "eeda85a3b21d493ca6bf9a8b3d034128"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "a3d4daadaf8ef02289cacc0f2740e3f9"
server_key_iv: "dfcb6f563eeb139fe89016f980c823b1"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.133601 11716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:42.151940 11716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:42.154379 11716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:42.154567 11716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:42.159018 11716 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:44751
I20260923 15:06:42.159070 11783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:44751 every 8 connection(s)
I20260923 15:06:42.159591 11716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:42.160272 11784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:42.162621 11784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128: Bootstrap starting.
I20260923 15:06:42.163242 11784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:42.163554 11784 log.cc:824] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:42.164273 11784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128: No bootstrap required, opened a new log
I20260923 15:06:42.165922 11784 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44751 } }
I20260923 15:06:42.166074 11784 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:42.166097 11784 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeda85a3b21d493ca6bf9a8b3d034128, State: Initialized, Role: FOLLOWER
I20260923 15:06:42.166276 11784 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44751 } }
I20260923 15:06:42.166358 11784 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:42.166391 11784 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:42.166442 11784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:42.167344 11784 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44751 } }
I20260923 15:06:42.167433 11784 leader_election.cc:302] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeda85a3b21d493ca6bf9a8b3d034128; no voters:
I20260923 15:06:42.167623 11784 leader_election.cc:288] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:42.167737 11787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:42.167877 11784 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:42.167888 11787 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [term 1 LEADER]: Becoming Leader. State: Replica: eeda85a3b21d493ca6bf9a8b3d034128, State: Running, Role: LEADER
I20260923 15:06:42.168052 11787 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44751 } }
I20260923 15:06:42.168247 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11716
I20260923 15:06:42.168358 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:42.168529 11788 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44751 } } }
I20260923 15:06:42.168576 8169 external_mini_cluster.cc:1428] Setting key 89fef08785a4da08a3e0e6250d6ac9d3
I20260923 15:06:42.168538 11789 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeda85a3b21d493ca6bf9a8b3d034128. Latest consensus state: current_term: 1 leader_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeda85a3b21d493ca6bf9a8b3d034128" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44751 } } }
I20260923 15:06:42.168727 11789 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:42.168658 11788 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128 [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:42.170807 11805 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:42.170866 11805 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:42.170909 11791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:42.171056 11791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:42.172646 11791 catalog_manager.cc:1347] Generated new cluster ID: 858870f90e2943698eb93ffe0fd3da74
I20260923 15:06:42.172698 11791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:42.179448 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.169040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54732 (local address 127.7.250.126:44751)
0923 15:06:42.169214 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:42.169224 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:42.169301 (+ 77us) server_negotiation.cc:407] Connection header received
0923 15:06:42.169440 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:42.169449 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:42.169517 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:42.169709 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:42.170310 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.171419 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.172465 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.173044 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.173176 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.175554 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:42.175558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:42.175558 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:42.175570 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:42.177791 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.178179 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.178182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.178182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.178327 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.178556 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.178557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.178558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.178823 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:42.179008 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:42.179081 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:06:42.179225 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
I20260923 15:06:42.190091 11791 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:42.190858 11791 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:42.200278 11791 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeda85a3b21d493ca6bf9a8b3d034128: Generated new TSK 0
I20260923 15:06:42.200557 11791 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:42.233915 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:44751
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:42.320446 11810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:42.320681 11810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:42.320721 11810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:42.321913 11810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:42.321983 11810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:42.322029 11810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:42.323580 11810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:44751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:42.323863 11810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:42.324108 11810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:42.329257 11816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:42.329252 11815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.329437 11810 server_base.cc:1034] running on GCE node
W20260923 15:06:42.329368 11818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.329775 11810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:42.330042 11810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:42.331220 11810 hybrid_clock.cc:648] HybridClock initialized: now 1790176002331183 us; error 52 us; skew 500 ppm
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:42.334045 11810 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:42.334617 11810 webserver.cc:460] Webserver started at http://127.7.250.65:44273/ using document root <none> and password file <none>
I20260923 15:06:42.334899 11810 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:42.334980 11810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:42.335137 11810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:42.336336 11810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "7b0b7967b3e04dea92707ebef3655fc2"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "ce35ebaeffc7e3c89019451f6217d4ee"
server_key_iv: "5f61fa427519532aeea2e881b427be51"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.336723 11810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "7b0b7967b3e04dea92707ebef3655fc2"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "ce35ebaeffc7e3c89019451f6217d4ee"
server_key_iv: "5f61fa427519532aeea2e881b427be51"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.338217 11810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:42.339210 11825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.339466 11810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:42.339567 11810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "7b0b7967b3e04dea92707ebef3655fc2"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "ce35ebaeffc7e3c89019451f6217d4ee"
server_key_iv: "5f61fa427519532aeea2e881b427be51"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.339658 11810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:42.350070 11810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:42.352320 11810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:42.352552 11810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:42.352808 11810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:42.353120 11810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:42.353176 11810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.353219 11810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:42.353241 11810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.359213 11810 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:40667
I20260923 15:06:42.359274 11938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:40667 every 8 connection(s)
I20260923 15:06:42.359668 11810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:42.369567 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.360528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:60635 (local address 127.7.250.126:44751)
0923 15:06:42.360599 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:42.360602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:42.360774 (+ 172us) server_negotiation.cc:407] Connection header received
0923 15:06:42.361766 (+ 992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:42.361768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:42.361786 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:42.361840 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:42.362555 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.363074 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.364107 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.364350 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.364439 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.366696 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:42.366697 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:42.366698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:42.366706 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:42.368104 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.368503 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.368504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.368504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.368561 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.368793 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.368794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.368794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.368982 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:42.369038 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:42.369291 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 15:06:42.369460 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:06:42.369604 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.360399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44751 (local address 127.7.250.65:60635)
0923 15:06:42.360625 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:42.360671 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:42.361634 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:42.361920 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:42.361924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:42.362186 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:42.362459 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:42.362463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.363176 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.363178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:42.363999 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:42.364001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.364452 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.364453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:42.364588 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.365066 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:42.365074 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:42.366567 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:42.368189 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:42.368196 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:42.368198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:42.368415 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:42.368632 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:42.368633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:42.368633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:42.368733 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:42.369050 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:42.369056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:42.369175 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 15:06:42.369263 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 15:06:42.369431 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":118,"threads_started":1}
I20260923 15:06:42.369884 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11810
I20260923 15:06:42.369982 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:42.370143 11939 heartbeater.cc:347] Connected to a master server at 127.7.250.126:44751
I20260923 15:06:42.370182 8169 external_mini_cluster.cc:1428] Setting key e41fc184d5edc9e2ba336f35483dfec4
I20260923 15:06:42.370239 11939 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:42.370501 11939 heartbeater.cc:511] Master 127.7.250.126:44751 requested a full tablet report, sending...
I20260923 15:06:42.371111 11747 ts_manager.cc:194] Registered new tserver with Master: 7b0b7967b3e04dea92707ebef3655fc2 (127.7.250.65:40667)
I20260923 15:06:42.372048 11747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:60635
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:42.397605 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:44751
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:42.478338 11944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:42.478582 11944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:42.478623 11944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:42.479779 11944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:42.479851 11944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:42.479893 11944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:42.481526 11944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:44751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:42.481794 11944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:42.482039 11944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:42.487214 11950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.487512 11944 server_base.cc:1034] running on GCE node
W20260923 15:06:42.487196 11952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:42.487200 11949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.487908 11944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:42.488193 11944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:42.489476 11944 hybrid_clock.cc:648] HybridClock initialized: now 1790176002489467 us; error 56 us; skew 500 ppm
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:42.492511 11944 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:42.493012 11944 webserver.cc:460] Webserver started at http://127.7.250.66:42107/ using document root <none> and password file <none>
I20260923 15:06:42.493285 11944 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:42.493363 11944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:42.493530 11944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:42.494793 11944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "9df16989585a497e933bc2c606961359"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "ee6daf4f6b1ce07b762a4e78bf94f122"
server_key_iv: "88b053b4cd69be7e7c893d3565c64f97"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.495203 11944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "9df16989585a497e933bc2c606961359"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "ee6daf4f6b1ce07b762a4e78bf94f122"
server_key_iv: "88b053b4cd69be7e7c893d3565c64f97"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.496703 11944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:06:42.497586 11959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.497815 11944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:42.497924 11944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "9df16989585a497e933bc2c606961359"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "ee6daf4f6b1ce07b762a4e78bf94f122"
server_key_iv: "88b053b4cd69be7e7c893d3565c64f97"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.498031 11944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:42.510383 11944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:42.512562 11944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:42.512732 11944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:42.513005 11944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:42.513329 11944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:42.513386 11944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.513427 11944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:42.513451 11944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.519411 11944 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:43915
I20260923 15:06:42.519490 12072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:43915 every 8 connection(s)
I20260923 15:06:42.519855 11944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:06:42.522907 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 11944
I20260923 15:06:42.523094 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:42.523300 8169 external_mini_cluster.cc:1428] Setting key c44785654136ca515c00645295bedb08
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:42.530580 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.520705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:58031 (local address 127.7.250.126:44751)
0923 15:06:42.520751 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:42.520753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:42.520984 (+ 231us) server_negotiation.cc:407] Connection header received
0923 15:06:42.521923 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:42.521925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:42.521937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:42.522012 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:42.522708 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.523659 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.524921 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.525286 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.525369 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.527624 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:42.527625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:42.527625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:42.527634 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:42.529078 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.529508 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.529509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.529510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.529567 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.529820 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.529821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.529821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.529939 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:42.530012 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:42.530333 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 15:06:42.530426 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:42.530771 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.520699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44751 (local address 127.7.250.66:58031)
0923 15:06:42.520853 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:42.520889 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:42.521814 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:42.522068 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:42.522071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:42.522414 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:42.522616 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:42.522621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.523777 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.523778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:42.524826 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:42.524829 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.525375 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.525376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:42.525472 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.526072 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:42.526080 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:42.527491 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:42.529175 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:42.529183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:42.529185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:42.529431 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:42.529637 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:42.529638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:42.529638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:42.529743 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:42.530023 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:42.530029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:42.530172 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 15:06:42.530326 (+ 154us) client_negotiation.cc:240] Negotiation successful
0923 15:06:42.530492 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":94,"threads_started":1}
I20260923 15:06:42.531266 12073 heartbeater.cc:347] Connected to a master server at 127.7.250.126:44751
I20260923 15:06:42.531365 12073 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:42.531563 12073 heartbeater.cc:511] Master 127.7.250.126:44751 requested a full tablet report, sending...
I20260923 15:06:42.531989 11747 ts_manager.cc:194] Registered new tserver with Master: 9df16989585a497e933bc2c606961359 (127.7.250.66:43915)
I20260923 15:06:42.532578 11747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:58031
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:42.553135 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:44751
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:42.634372 12078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:42.634670 12078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:42.634712 12078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:42.635908 12078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:42.635957 12078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:42.635984 12078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:42.637732 12078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:44751
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:42.637986 12078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:42.638223 12078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:42.643517 12083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:42.643481 12086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:42.643486 12084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:42.643564 12078 server_base.cc:1034] running on GCE node
I20260923 15:06:42.643880 12078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:42.644148 12078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:42.645320 12078 hybrid_clock.cc:648] HybridClock initialized: now 1790176002645304 us; error 26 us; skew 500 ppm
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:42.648138 12078 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:42.648649 12078 webserver.cc:460] Webserver started at http://127.7.250.67:43711/ using document root <none> and password file <none>
I20260923 15:06:42.648913 12078 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:42.648990 12078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:42.649147 12078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:42.650460 12078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "da6e2f08878c43098f56151a837fee83"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "126820d9cf3f35996e251bfb4d7e2e2a"
server_key_iv: "7fb607d85722bfe8297477cf0338a6a3"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.650844 12078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "da6e2f08878c43098f56151a837fee83"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "126820d9cf3f35996e251bfb4d7e2e2a"
server_key_iv: "7fb607d85722bfe8297477cf0338a6a3"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.652275 12078 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:42.653057 12093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.653244 12078 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:42.653350 12078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "da6e2f08878c43098f56151a837fee83"
format_stamp: "Formatted at 2026-09-23 15:06:42 on dist-test-slave-bb3x"
server_key: "126820d9cf3f35996e251bfb4d7e2e2a"
server_key_iv: "7fb607d85722bfe8297477cf0338a6a3"
server_key_version: "encryptionkey@0"
I20260923 15:06:42.653522 12078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:42.660161 12078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:42.662453 12078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:42.662611 12078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:42.662835 12078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:42.663146 12078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:42.663199 12078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.663241 12078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:42.663267 12078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:42.669341 12078 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:42575
I20260923 15:06:42.669392 12206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:42575 every 8 connection(s)
I20260923 15:06:42.669719 12078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:42.678197 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12078
I20260923 15:06:42.678372 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:42.678582 8169 external_mini_cluster.cc:1428] Setting key 38420af3e5151fb3440f31d167540400
I20260923 15:06:42.679850 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.670620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:44659 (local address 127.7.250.126:44751)
0923 15:06:42.670665 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:42.670668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:42.670883 (+ 215us) server_negotiation.cc:407] Connection header received
0923 15:06:42.671894 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:42.671896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:42.671910 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:42.671960 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:42.672649 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.673176 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.674190 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.674462 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.674537 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.676738 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:42.676740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:42.676740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:42.676748 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:42.678089 (+ 1341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.678690 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.678691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.678692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.678769 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.679129 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.679130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.679131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.679293 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:42.679376 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:42.679707 (+ 331us) server_negotiation.cc:299] Negotiation successful
0923 15:06:42.679785 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:42.680091 12209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.670597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44751 (local address 127.7.250.67:44659)
0923 15:06:42.670758 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:42.670795 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:42.671769 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:42.672029 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:42.672033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:42.672302 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:42.672547 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:42.672551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.673280 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.673281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:42.674092 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:42.674094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.674559 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.674560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:42.674687 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.675157 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:42.675166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:42.676637 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:42.678194 (+ 1557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:42.678201 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:42.678203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:42.678566 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:42.678895 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:42.678896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:42.678896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:42.679056 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:42.679381 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:42.679388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:42.679562 (+ 174us) client_negotiation.cc:769] Sending connection context
0923 15:06:42.679671 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:06:42.679824 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260923 15:06:42.680536 12207 heartbeater.cc:347] Connected to a master server at 127.7.250.126:44751
I20260923 15:06:42.680660 12207 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:42.680843 12207 heartbeater.cc:511] Master 127.7.250.126:44751 requested a full tablet report, sending...
I20260923 15:06:42.681208 11747 ts_manager.cc:194] Registered new tserver with Master: da6e2f08878c43098f56151a837fee83 (127.7.250.67:42575)
I20260923 15:06:42.681664 11747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:44659
I20260923 15:06:42.690873 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:42.715305 12214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.707347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.7.250.65:40667)
0923 15:06:42.707560 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:42.707565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:42.707574 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:42.707598 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:42.707600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:42.707614 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:42.707675 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:42.708245 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.708985 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.709753 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.710104 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.710162 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.712454 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:42.712462 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:42.712462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:42.712469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:42.714160 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.714550 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.714553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.714553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.714611 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.714767 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.714768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.714768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.714910 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:42.715025 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:42.715086 (+ 61us) server_negotiation.cc:299] Negotiation successful
0923 15:06:42.715154 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":114,"threads_started":1}
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[11703](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:42.723830 11794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:42.716641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.7.250.126:44751)
0923 15:06:42.716692 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:42.716694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:42.716704 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:42.716749 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:42.716750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:42.716759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:42.716812 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:42.717311 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.717803 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.718563 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:42.718775 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:42.718839 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:42.721208 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:42.721209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:42.721209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:42.721216 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:42.722652 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.722999 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.723000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.723001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.723058 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:42.723309 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:42.723310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:42.723310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:42.723469 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:42.723549 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:42.723698 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 15:06:42.723764 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:06:42.724136 11747 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54740
I20260923 15:06:42.724691 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11810
I20260923 15:06:42.729072 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11944
I20260923 15:06:42.733569 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12078
I20260923 15:06:42.737962 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 11716
2026-09-23T15:06:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1897 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[12219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[12219](info): set up 2 sockets
Sep 23 15:06:42 dist-test-slave-bb3x krb5kdc[12219](info): commencing operation
krb5kdc: starting...
W20260923 15:06:43.812538 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:43 dist-test-slave-bb3x krb5kdc[12219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:43.942250 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:36345
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:36345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:44.023417 12232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:44.023626 12232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:44.023649 12232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:44.024739 12232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:44.024776 12232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:44.024791 12232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:44.024802 12232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:44.024814 12232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:44.026288 12232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:36345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:36345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:44.026546 12232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:44.026762 12232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:44.031674 12240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.031708 12237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.031700 12238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:44.031999 12232 server_base.cc:1034] running on GCE node
I20260923 15:06:44.032272 12232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:44.032660 12232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:44.033830 12232 hybrid_clock.cc:648] HybridClock initialized: now 1790176004033805 us; error 41 us; skew 500 ppm
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:44.036743 12232 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:44.037151 12232 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:44.037223 12232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:44.037358 12232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:44.038525 12232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "93847042440e47c9a4c9c3104ab21ece"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "c5bf4f164ccacd4cdfd105026f9febca"
server_key_iv: "37a71cbe126445b08739b0bf599fe25e"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.038875 12232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "93847042440e47c9a4c9c3104ab21ece"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "c5bf4f164ccacd4cdfd105026f9febca"
server_key_iv: "37a71cbe126445b08739b0bf599fe25e"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.040169 12232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:44.040992 12246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.041188 12232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:44.041287 12232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "93847042440e47c9a4c9c3104ab21ece"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "c5bf4f164ccacd4cdfd105026f9febca"
server_key_iv: "37a71cbe126445b08739b0bf599fe25e"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.041366 12232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:44.064193 12232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:44.066390 12232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:44.066579 12232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:44.071077 12232 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:36345
I20260923 15:06:44.071105 12298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:36345 every 8 connection(s)
I20260923 15:06:44.071439 12232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:44.072502 12299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:44.074733 12299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece: Bootstrap starting.
I20260923 15:06:44.075258 12299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:44.075511 12299 log.cc:824] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:44.076133 12299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece: No bootstrap required, opened a new log
I20260923 15:06:44.077430 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12232
I20260923 15:06:44.077582 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:44.077796 8169 external_mini_cluster.cc:1428] Setting key ef95653c66e0e766f5fb2f2845b5c1e0
I20260923 15:06:44.077817 12299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93847042440e47c9a4c9c3104ab21ece" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36345 } }
I20260923 15:06:44.078032 12299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:44.078086 12299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93847042440e47c9a4c9c3104ab21ece, State: Initialized, Role: FOLLOWER
I20260923 15:06:44.078234 12299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93847042440e47c9a4c9c3104ab21ece" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36345 } }
I20260923 15:06:44.078346 12299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:44.078382 12299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:44.078470 12299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:44.079299 12299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93847042440e47c9a4c9c3104ab21ece" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36345 } }
I20260923 15:06:44.079435 12299 leader_election.cc:302] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93847042440e47c9a4c9c3104ab21ece; no voters:
I20260923 15:06:44.079591 12299 leader_election.cc:288] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:44.079620 12304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:44.079720 12304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [term 1 LEADER]: Becoming Leader. State: Replica: 93847042440e47c9a4c9c3104ab21ece, State: Running, Role: LEADER
I20260923 15:06:44.079856 12299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:44.079864 12304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93847042440e47c9a4c9c3104ab21ece" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36345 } }
I20260923 15:06:44.080535 12306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93847042440e47c9a4c9c3104ab21ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93847042440e47c9a4c9c3104ab21ece" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36345 } } }
I20260923 15:06:44.080639 12306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:44.080545 12305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93847042440e47c9a4c9c3104ab21ece. Latest consensus state: current_term: 1 leader_uuid: "93847042440e47c9a4c9c3104ab21ece" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93847042440e47c9a4c9c3104ab21ece" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36345 } } }
I20260923 15:06:44.080957 12305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:44.081130 12311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:44.081789 12311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:44.083413 12311 catalog_manager.cc:1347] Generated new cluster ID: 9097da65344445ab86c5225d63002244
I20260923 15:06:44.083468 12311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:44.089437 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.078361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57726 (local address 127.7.250.126:36345)
0923 15:06:44.078565 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.078576 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.078611 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:06:44.078753 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.078759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.078820 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.078998 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:44.079846 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.081573 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.082773 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.083355 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.083467 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.086001 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:44.086005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:44.086005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:44.086028 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:44.087888 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.088270 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.088274 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.088274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.088376 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.088589 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.088590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.088590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.088829 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:44.089007 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.089080 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.089225 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260923 15:06:44.097275 12311 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:44.098184 12311 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:44.103760 12311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece: Generated new TSK 0
I20260923 15:06:44.103965 12311 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:44.137955 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36345
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:44.236125 12325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:44.236361 12325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:44.236402 12325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:44.237553 12325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:44.237622 12325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:44.237669 12325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:44.239274 12325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:44.239547 12325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:44.239779 12325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:44.244890 12330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.245015 12333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.245025 12331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:44.245289 12325 server_base.cc:1034] running on GCE node
I20260923 15:06:44.245501 12325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:44.245765 12325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:44.246948 12325 hybrid_clock.cc:648] HybridClock initialized: now 1790176004246930 us; error 27 us; skew 500 ppm
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:44.249894 12325 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:44.250249 12325 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:44.250355 12325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:44.250599 12325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:44.251683 12325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "e1b8903da29345a48fd4faaace4897c4"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "b6b6e53d980bcab70f2ffc0c73b4fe3e"
server_key_iv: "0881fa5e3394d404b2b34496ead2f268"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.252051 12325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "e1b8903da29345a48fd4faaace4897c4"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "b6b6e53d980bcab70f2ffc0c73b4fe3e"
server_key_iv: "0881fa5e3394d404b2b34496ead2f268"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.253553 12325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:44.254494 12339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.254650 12325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:44.254739 12325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "e1b8903da29345a48fd4faaace4897c4"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "b6b6e53d980bcab70f2ffc0c73b4fe3e"
server_key_iv: "0881fa5e3394d404b2b34496ead2f268"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.254810 12325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:44.276526 12325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:44.278942 12325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:44.279101 12325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:44.279373 12325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:44.279757 12325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:44.279824 12325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.279867 12325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:44.279913 12325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.286172 12325 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:42051
I20260923 15:06:44.286243 12452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:42051 every 8 connection(s)
I20260923 15:06:44.286571 12325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:44.294760 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12325
I20260923 15:06:44.294880 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:44.295104 8169 external_mini_cluster.cc:1428] Setting key 9c9ccf17b221e09d2505d626599ed414
I20260923 15:06:44.297206 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.287371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:59433 (local address 127.7.250.126:36345)
0923 15:06:44.287450 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.287453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.287637 (+ 184us) server_negotiation.cc:407] Connection header received
0923 15:06:44.288162 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.288163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.288177 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.288236 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:44.289251 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.289768 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.290943 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.291152 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.291213 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.293623 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:44.293624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:44.293625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:44.293637 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:44.295649 (+ 2012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.296128 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.296130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.296130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.296195 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.296496 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.296497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.296497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.296629 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:44.296714 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.297014 (+ 300us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.297107 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 15:06:44.297422 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.287324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36345 (local address 127.7.250.65:59433)
0923 15:06:44.287500 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.287540 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.288035 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.288306 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.288310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.288571 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:44.289145 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.289151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.289915 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.289916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.290844 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.290846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.291239 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.291240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.291353 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.291919 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:44.291928 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:44.293439 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:44.295763 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:44.295772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:44.295775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:44.296022 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:44.296292 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:44.296292 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:44.296293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:44.296417 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:44.296703 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:44.296711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:44.296869 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.296991 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.297161 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260923 15:06:44.297933 12453 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36345
I20260923 15:06:44.298058 12453 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:44.298220 12453 heartbeater.cc:511] Master 127.7.250.126:36345 requested a full tablet report, sending...
I20260923 15:06:44.298864 12263 ts_manager.cc:194] Registered new tserver with Master: e1b8903da29345a48fd4faaace4897c4 (127.7.250.65:42051)
I20260923 15:06:44.299831 12263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:59433
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:44.321946 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36345
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:44.403405 12458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:44.403678 12458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:44.403719 12458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:44.404914 12458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:44.404995 12458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:44.405043 12458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:44.406641 12458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:44.406926 12458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:44.407171 12458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:44.412535 12464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.412573 12463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.412572 12466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:44.412616 12458 server_base.cc:1034] running on GCE node
I20260923 15:06:44.412956 12458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:44.413198 12458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:44.414325 12458 hybrid_clock.cc:648] HybridClock initialized: now 1790176004414318 us; error 35 us; skew 500 ppm
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:44.417172 12458 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:44.417668 12458 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:44.417758 12458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:44.417933 12458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:44.419166 12458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "ccdda9c765634e7481ecc2dd6fcf517d"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "15521d82b8982268bca09305de661e2f"
server_key_iv: "d274451e756108f316b28323e873af10"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.419579 12458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "ccdda9c765634e7481ecc2dd6fcf517d"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "15521d82b8982268bca09305de661e2f"
server_key_iv: "d274451e756108f316b28323e873af10"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.421005 12458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:44.421895 12472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.422076 12458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:44.422186 12458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "ccdda9c765634e7481ecc2dd6fcf517d"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "15521d82b8982268bca09305de661e2f"
server_key_iv: "d274451e756108f316b28323e873af10"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.422299 12458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:44.441855 12458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:44.444025 12458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:44.444206 12458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:44.444444 12458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:44.444852 12458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:44.444921 12458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.444957 12458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:44.444983 12458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.450685 12458 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:33193
I20260923 15:06:44.450721 12585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:33193 every 8 connection(s)
I20260923 15:06:44.451014 12458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:44.457232 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12458
I20260923 15:06:44.457352 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:44.457616 8169 external_mini_cluster.cc:1428] Setting key 3f7837a892b20842968ab92ff44c3405
I20260923 15:06:44.460698 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.451713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:43939 (local address 127.7.250.126:36345)
0923 15:06:44.451755 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.451757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.451963 (+ 206us) server_negotiation.cc:407] Connection header received
0923 15:06:44.452532 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.452534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.452546 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.452622 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:44.453555 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.454055 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.455138 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.455336 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.455429 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.457490 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:44.457492 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:44.457492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:44.457500 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:44.459240 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.459631 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.459633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.459633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.459703 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.460023 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.460023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.460024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.460127 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:44.460180 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.460541 (+ 361us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.460628 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 15:06:44.460973 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.451683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36345 (local address 127.7.250.66:43939)
0923 15:06:44.451835 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.451876 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.452368 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.452696 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.452700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.452961 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:44.453446 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.453450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.454146 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.454147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.455031 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.455033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.455437 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.455438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.455573 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.455989 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:44.455997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:44.457361 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:44.459329 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:44.459335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:44.459337 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:44.459560 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:44.459778 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:44.459779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:44.459779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:44.459940 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:44.460191 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:44.460197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:44.460364 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.460483 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.460668 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260923 15:06:44.461437 12586 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36345
I20260923 15:06:44.461551 12586 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:44.461711 12586 heartbeater.cc:511] Master 127.7.250.126:36345 requested a full tablet report, sending...
I20260923 15:06:44.462109 12263 ts_manager.cc:194] Registered new tserver with Master: ccdda9c765634e7481ecc2dd6fcf517d (127.7.250.66:33193)
I20260923 15:06:44.462538 12263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:43939
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:44.486392 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36345
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:44.571022 12591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:44.571213 12591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:44.571237 12591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:44.572388 12591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:44.572429 12591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:44.572458 12591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:44.573954 12591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:44.574195 12591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:44.574435 12591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:44.579634 12596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.579604 12597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.579625 12599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:44.579859 12591 server_base.cc:1034] running on GCE node
I20260923 15:06:44.580143 12591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:44.580432 12591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:44.581589 12591 hybrid_clock.cc:648] HybridClock initialized: now 1790176004581573 us; error 39 us; skew 500 ppm
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:44.584168 12591 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:44.584568 12591 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:44.584641 12591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:44.584779 12591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:44.585909 12591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "cf7df4be0c704156a1295ab463023bff"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "52c93f98e0f8561891e3a9d849388334"
server_key_iv: "1d26c19936091e897b2d82644620b713"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.586350 12591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "cf7df4be0c704156a1295ab463023bff"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "52c93f98e0f8561891e3a9d849388334"
server_key_iv: "1d26c19936091e897b2d82644620b713"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.587643 12591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:44.588402 12605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.588573 12591 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:44.588672 12591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "cf7df4be0c704156a1295ab463023bff"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "52c93f98e0f8561891e3a9d849388334"
server_key_iv: "1d26c19936091e897b2d82644620b713"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.588763 12591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:44.602903 12591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:44.605132 12591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:44.605293 12591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:44.605530 12591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:44.605841 12591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:44.605896 12591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.605939 12591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:44.605973 12591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.611982 12591 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:41711
I20260923 15:06:44.612018 12718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:41711 every 8 connection(s)
I20260923 15:06:44.612314 12591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:06:44 dist-test-slave-bb3x krb5kdc[12219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176004, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:44.621690 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12591
I20260923 15:06:44.621790 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:44.621832 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.613068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:53383 (local address 127.7.250.126:36345)
0923 15:06:44.613166 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.613168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.613360 (+ 192us) server_negotiation.cc:407] Connection header received
0923 15:06:44.613783 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.613784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.613795 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.613858 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:44.614840 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.615349 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.616475 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.616683 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.616755 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.619084 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:44.619086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:44.619086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:44.619093 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:44.620530 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.620892 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.620893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.620893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.620969 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.621178 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.621179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.621179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.621282 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:44.621332 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.621630 (+ 298us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.621722 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 15:06:44.621968 12721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.612994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36345 (local address 127.7.250.67:53383)
0923 15:06:44.613232 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.613262 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.613700 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.613917 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.613920 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.614173 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:44.614728 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.614733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.615483 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.615484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.616372 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.616374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.616821 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.616822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.616935 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.617519 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:44.617528 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:44.618999 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:44.620610 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:44.620616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:44.620618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:44.620825 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:44.621029 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:44.621030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:44.621030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:44.621123 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:44.621335 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:44.621338 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:44.621450 (+ 112us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.621574 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.621697 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":82,"threads_started":1}
I20260923 15:06:44.622045 8169 external_mini_cluster.cc:1428] Setting key 78e315b2cad27c32bbc983f26312a91e
I20260923 15:06:44.622486 12719 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36345
I20260923 15:06:44.622610 12719 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:44.622777 12719 heartbeater.cc:511] Master 127.7.250.126:36345 requested a full tablet report, sending...
I20260923 15:06:44.623172 12263 ts_manager.cc:194] Registered new tserver with Master: cf7df4be0c704156a1295ab463023bff (127.7.250.67:41711)
I20260923 15:06:44.623554 12263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:53383
I20260923 15:06:44.634611 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:06:44.642207 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.636094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.7.250.126:36345)
0923 15:06:44.636155 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.636157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.636226 (+ 69us) server_negotiation.cc:407] Connection header received
0923 15:06:44.636265 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.636265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.636274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.636330 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:44.636916 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.637429 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.638182 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.638434 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.638508 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.639703 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:44.639704 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:44.639704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:44.639711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:44.641042 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.641369 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.641370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.641371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.641434 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:44.641675 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:44.641676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:44.641677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:44.641828 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:44.641906 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.642048 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.642111 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:06:44.643985 12263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:44.645587 12263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:44.652206 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.648125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36332 (local address 127.7.250.65:42051)
0923 15:06:44.648293 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.648298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.648425 (+ 127us) server_negotiation.cc:407] Connection header received
0923 15:06:44.648527 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.648529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.648545 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.648651 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:44.648852 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.649688 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.650919 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.651681 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.651731 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.651787 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.652075 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.652116 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260923 15:06:44.652225 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.648022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:42051 (local address 127.0.0.1:36332)
0923 15:06:44.648345 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.648355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.648449 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.648668 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.648672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.648675 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:44.648759 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.648761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.649784 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.649785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.650811 (+ 1026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.650814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.651851 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.651852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.651938 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.651955 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.652051 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.652111 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":36,"thread_start_us":64,"threads_started":1}
I20260923 15:06:44.652393 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.648022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:41711 (local address 127.0.0.1:44624)
0923 15:06:44.648140 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.648182 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.648243 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.648751 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.648752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.648753 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:44.648793 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.648794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.649928 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.649929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.651123 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.651127 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.651971 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.651972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.652051 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.652056 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.652111 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.652150 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":90,"threads_started":1}
I20260923 15:06:44.653059 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.648379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.7.250.67:41711)
0923 15:06:44.648563 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.648568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.648579 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:44.648618 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.648621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.648637 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.648751 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:44.648859 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.649813 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.651214 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.651885 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.651960 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.652014 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.652904 (+ 890us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.652948 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":125,"threads_started":1}
I20260923 15:06:44.653071 12387 tablet_service.cc:1467] Processing CreateTablet for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 (DEFAULT_TABLE table=test-table [id=bf259f8043804a789f15ab5beae446b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:44.653431 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.648235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33193 (local address 127.0.0.1:43306)
0923 15:06:44.648920 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.648929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.648977 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.650085 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.650086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.650088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:44.650123 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.650124 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.651373 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.651374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.652162 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.652164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.653073 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.653074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.653256 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.653261 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.653314 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.653351 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":73,"threads_started":1}
I20260923 15:06:44.653475 12387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:44.653699 12735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.648704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.7.250.66:33193)
0923 15:06:44.649831 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.649834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.649842 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:44.649898 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.649900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.649918 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.649999 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:44.650199 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.651263 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.652251 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.652979 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.653056 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.653112 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.653581 (+ 469us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.653627 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1095,"thread_start_us":82,"threads_started":1}
I20260923 15:06:44.654431 12520 tablet_service.cc:1467] Processing CreateTablet for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 (DEFAULT_TABLE table=test-table [id=bf259f8043804a789f15ab5beae446b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:44.654767 12520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:44.654850 12653 tablet_service.cc:1467] Processing CreateTablet for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 (DEFAULT_TABLE table=test-table [id=bf259f8043804a789f15ab5beae446b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:44.655180 12653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:44.656903 12736 tablet_bootstrap.cc:492] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: Bootstrap starting.
I20260923 15:06:44.657688 12736 tablet_bootstrap.cc:654] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:44.658015 12737 tablet_bootstrap.cc:492] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Bootstrap starting.
I20260923 15:06:44.658420 12736 log.cc:824] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:44.658578 12738 tablet_bootstrap.cc:492] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Bootstrap starting.
I20260923 15:06:44.658685 12737 tablet_bootstrap.cc:654] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:44.659132 12736 tablet_bootstrap.cc:492] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: No bootstrap required, opened a new log
I20260923 15:06:44.659224 12736 ts_tablet_manager.cc:1397] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:06:44.659318 12738 tablet_bootstrap.cc:654] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:44.659294 12737 log.cc:824] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:44.660029 12738 log.cc:824] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:44.660147 12737 tablet_bootstrap.cc:492] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: No bootstrap required, opened a new log
I20260923 15:06:44.660218 12737 ts_tablet_manager.cc:1397] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:06:44.660766 12738 tablet_bootstrap.cc:492] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: No bootstrap required, opened a new log
I20260923 15:06:44.660845 12738 ts_tablet_manager.cc:1397] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:06:44.661394 12736 raft_consensus.cc:348] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.661581 12736 raft_consensus.cc:374] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:44.661621 12736 raft_consensus.cc:729] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1b8903da29345a48fd4faaace4897c4, State: Initialized, Role: FOLLOWER
I20260923 15:06:44.661741 12736 consensus_queue.cc:260] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.661798 12737 raft_consensus.cc:348] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.661979 12737 raft_consensus.cc:374] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:44.662007 12736 ts_tablet_manager.cc:1428] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:06:44.662026 12737 raft_consensus.cc:729] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdda9c765634e7481ecc2dd6fcf517d, State: Initialized, Role: FOLLOWER
I20260923 15:06:44.662127 12453 heartbeater.cc:503] Master 127.7.250.126:36345 was elected leader, sending a full tablet report...
I20260923 15:06:44.662148 12737 consensus_queue.cc:260] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.662449 12737 ts_tablet_manager.cc:1428] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260923 15:06:44.662634 12586 heartbeater.cc:503] Master 127.7.250.126:36345 was elected leader, sending a full tablet report...
I20260923 15:06:44.662909 12738 raft_consensus.cc:348] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.663069 12738 raft_consensus.cc:374] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:44.663111 12738 raft_consensus.cc:729] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf7df4be0c704156a1295ab463023bff, State: Initialized, Role: FOLLOWER
I20260923 15:06:44.663221 12738 consensus_queue.cc:260] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.663491 12738 ts_tablet_manager.cc:1428] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:44.663501 12719 heartbeater.cc:503] Master 127.7.250.126:36345 was elected leader, sending a full tablet report...
W20260923 15:06:44.701586 12587 tablet.cc:2369] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:44.736078 12742 raft_consensus.cc:482] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:44.736215 12742 raft_consensus.cc:504] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.736683 12742 leader_election.cc:288] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf7df4be0c704156a1295ab463023bff (127.7.250.67:41711), ccdda9c765634e7481ecc2dd6fcf517d (127.7.250.66:33193)
I20260923 15:06:44.740080 12455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.736947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:41711 (local address 127.7.250.65:41477)
0923 15:06:44.737015 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.737030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.737123 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.737300 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.737301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.737303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:44.737361 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.737363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.738154 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.738156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.739110 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.739112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.739762 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.739763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.739845 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.739849 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.739942 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.739996 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:06:44.740080 12734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.737063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:41477 (local address 127.7.250.67:41711)
0923 15:06:44.737140 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.737143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.737155 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:44.737203 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.737204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.737216 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.737283 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:44.737451 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.738037 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.739221 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.739660 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.739762 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.739772 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.739951 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.739995 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 15:06:44.740236 12735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.737221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:57193 (local address 127.7.250.66:33193)
0923 15:06:44.737296 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.737298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.737404 (+ 106us) server_negotiation.cc:407] Connection header received
0923 15:06:44.737528 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.737529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.737541 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.737601 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:44.737778 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.738334 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.739414 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.739839 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.739939 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.739948 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.740128 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.740171 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 15:06:44.740242 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.737103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33193 (local address 127.7.250.65:57193)
0923 15:06:44.737315 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.737326 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:44.737410 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:44.737604 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:44.737605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:44.737607 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:44.737696 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.737697 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.738446 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.738447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.739302 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:44.739304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.739939 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.739940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:44.740032 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.740036 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:44.740107 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 15:06:44.740153 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":135,"threads_started":1}
I20260923 15:06:44.740514 12673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df5fc0800ba4d1bb520b2cbbdb13fd1" candidate_uuid: "e1b8903da29345a48fd4faaace4897c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7df4be0c704156a1295ab463023bff" is_pre_election: true
I20260923 15:06:44.740731 12673 raft_consensus.cc:2454] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1b8903da29345a48fd4faaace4897c4 in term 0.
I20260923 15:06:44.741057 12342 leader_election.cc:302] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf7df4be0c704156a1295ab463023bff, e1b8903da29345a48fd4faaace4897c4; no voters:
I20260923 15:06:44.741166 12540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df5fc0800ba4d1bb520b2cbbdb13fd1" candidate_uuid: "e1b8903da29345a48fd4faaace4897c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" is_pre_election: true
I20260923 15:06:44.741211 12742 raft_consensus.cc:2781] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:44.741268 12742 raft_consensus.cc:482] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:44.741294 12742 raft_consensus.cc:3037] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:44.741334 12540 raft_consensus.cc:2454] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1b8903da29345a48fd4faaace4897c4 in term 0.
I20260923 15:06:44.742097 12742 raft_consensus.cc:504] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.742298 12742 leader_election.cc:288] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [CANDIDATE]: Term 1 election: Requested vote from peers cf7df4be0c704156a1295ab463023bff (127.7.250.67:41711), ccdda9c765634e7481ecc2dd6fcf517d (127.7.250.66:33193)
I20260923 15:06:44.742623 12540 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df5fc0800ba4d1bb520b2cbbdb13fd1" candidate_uuid: "e1b8903da29345a48fd4faaace4897c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccdda9c765634e7481ecc2dd6fcf517d"
I20260923 15:06:44.742621 12673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2df5fc0800ba4d1bb520b2cbbdb13fd1" candidate_uuid: "e1b8903da29345a48fd4faaace4897c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf7df4be0c704156a1295ab463023bff"
I20260923 15:06:44.742714 12673 raft_consensus.cc:3037] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:44.742714 12540 raft_consensus.cc:3037] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:44.743702 12673 raft_consensus.cc:2454] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1b8903da29345a48fd4faaace4897c4 in term 1.
I20260923 15:06:44.743702 12540 raft_consensus.cc:2454] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1b8903da29345a48fd4faaace4897c4 in term 1.
I20260923 15:06:44.743920 12342 leader_election.cc:302] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccdda9c765634e7481ecc2dd6fcf517d, e1b8903da29345a48fd4faaace4897c4; no voters:
I20260923 15:06:44.744027 12742 raft_consensus.cc:2781] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:44.744226 12742 raft_consensus.cc:686] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [term 1 LEADER]: Becoming Leader. State: Replica: e1b8903da29345a48fd4faaace4897c4, State: Running, Role: LEADER
I20260923 15:06:44.744347 12742 consensus_queue.cc:237] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } }
I20260923 15:06:44.745391 12262 catalog_manager.cc:5495] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1b8903da29345a48fd4faaace4897c4 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "e1b8903da29345a48fd4faaace4897c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1b8903da29345a48fd4faaace4897c4" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 42051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:44.769887 12732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.767396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36342 (local address 127.7.250.65:42051)
0923 15:06:44.767510 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.767512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.767519 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:44.767556 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.767557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.767574 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.767657 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:44.767835 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.768318 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.769096 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:44.769295 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:44.769374 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:44.769531 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:44.769634 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:44.769735 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.769754 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.769775 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:06:44.772158 12540 raft_consensus.cc:1261] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 1 FOLLOWER]: Refusing update from remote peer e1b8903da29345a48fd4faaace4897c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:44.772158 12673 raft_consensus.cc:1261] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 1 FOLLOWER]: Refusing update from remote peer e1b8903da29345a48fd4faaace4897c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:44.772565 12742 consensus_queue.cc:1035] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf7df4be0c704156a1295ab463023bff" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:44.772728 12746 consensus_queue.cc:1035] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccdda9c765634e7481ecc2dd6fcf517d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:44.775089 12749 mvcc.cc:205] Tried to move back new op lower bound from 7332560915543289856 to 7332560915433713664. Current Snapshot: MvccSnapshot[applied={T|T < 7332560915543289856}]
I20260923 15:06:44.776037 12751 mvcc.cc:205] Tried to move back new op lower bound from 7332560915543289856 to 7332560915433713664. Current Snapshot: MvccSnapshot[applied={T|T < 7332560915543289856}]
I20260923 15:06:44.777931 12263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:44.778049 12263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:44.778887 12263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 93847042440e47c9a4c9c3104ab21ece: Sending DeleteTablet for 3 replicas of tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1
I20260923 15:06:44.779224 12653 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:44 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:44624
I20260923 15:06:44.779247 12520 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:44 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:43306
I20260923 15:06:44.779444 12387 tablet_service.cc:1514] Processing DeleteTablet for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:44 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:36332
I20260923 15:06:44.779598 12758 tablet_replica.cc:330] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: stopping tablet replica
I20260923 15:06:44.779683 12760 tablet_replica.cc:330] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P e1b8903da29345a48fd4faaace4897c4: stopping tablet replica
I20260923 15:06:44.779697 12758 raft_consensus.cc:2229] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:44.779798 12758 raft_consensus.cc:2258] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:44.779834 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12325
I20260923 15:06:44.781031 12759 tablet_replica.cc:330] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: stopping tablet replica
I20260923 15:06:44.781143 12759 raft_consensus.cc:2229] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:44.781250 12759 raft_consensus.cc:2258] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:44.781750 12758 ts_tablet_manager.cc:1905] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:44.782297 12759 ts_tablet_manager.cc:1905] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:44.783955 12759 ts_tablet_manager.cc:1918] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:44.783957 12758 ts_tablet_manager.cc:1918] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:44.784030 12758 log.cc:1196] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/2df5fc0800ba4d1bb520b2cbbdb13fd1
I20260923 15:06:44.784070 12759 log.cc:1196] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/2df5fc0800ba4d1bb520b2cbbdb13fd1
I20260923 15:06:44.784348 12758 ts_tablet_manager.cc:1939] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P ccdda9c765634e7481ecc2dd6fcf517d: Deleting consensus metadata
I20260923 15:06:44.784349 12759 ts_tablet_manager.cc:1939] T 2df5fc0800ba4d1bb520b2cbbdb13fd1 P cf7df4be0c704156a1295ab463023bff: Deleting consensus metadata
I20260923 15:06:44.784792 12249 catalog_manager.cc:4841] TS cf7df4be0c704156a1295ab463023bff (127.7.250.67:41711): tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 (table test-table [id=bf259f8043804a789f15ab5beae446b9]) successfully deleted
I20260923 15:06:44.784894 12249 catalog_manager.cc:4841] TS ccdda9c765634e7481ecc2dd6fcf517d (127.7.250.66:33193): tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1 (table test-table [id=bf259f8043804a789f15ab5beae446b9]) successfully deleted
W20260923 15:06:44.786386 12250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:42051 (error 108)
I20260923 15:06:44.786500 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12458
I20260923 15:06:44.787439 12731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.787266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:42051 (local address 127.0.0.1:36358)
0923 15:06:44.787343 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:44.787387 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:42051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 15:06:44.787607 12250 catalog_manager.cc:4584] TS e1b8903da29345a48fd4faaace4897c4 (127.7.250.65:42051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2df5fc0800ba4d1bb520b2cbbdb13fd1: Network error: Client connection negotiation failed: client connection to 127.7.250.65:42051: connect: Connection refused (error 111)
I20260923 15:06:44.791800 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12591
I20260923 15:06:44.796396 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12232
2026-09-23T15:06:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2058 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T15:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:44Z Disabled control of system clock
I20260923 15:06:44.823514 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42769
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:42769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 15:06:44.905036 12765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:44.905251 12765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:44.905273 12765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:44.906563 12765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:44.906606 12765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:44.906620 12765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:44.906632 12765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:44.906644 12765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:44.908113 12765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:42769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:44.908383 12765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:44.908699 12765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:44.913771 12773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.913765 12771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:44.913793 12770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:44.914183 12765 server_base.cc:1034] running on GCE node
I20260923 15:06:44.914464 12765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:44.914788 12765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:44.915983 12765 hybrid_clock.cc:648] HybridClock initialized: now 1790176004915999 us; error 58 us; skew 500 ppm
I20260923 15:06:44.917619 12765 webserver.cc:460] Webserver started at http://127.7.250.126:37857/ using document root <none> and password file <none>
I20260923 15:06:44.917963 12765 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:44.918022 12765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:44.918165 12765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:44.919649 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "9cfdc4793ac24daf9eca43b251baa125"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "5ad806181bdc47836b414980b1fada31"
server_key_iv: "ba2717cb6360c4b2cfa9adc4b8c4b242"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.920003 12765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "9cfdc4793ac24daf9eca43b251baa125"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "5ad806181bdc47836b414980b1fada31"
server_key_iv: "ba2717cb6360c4b2cfa9adc4b8c4b242"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.921262 12765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:44.922049 12779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:44.922241 12765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:44.922358 12765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "9cfdc4793ac24daf9eca43b251baa125"
format_stamp: "Formatted at 2026-09-23 15:06:44 on dist-test-slave-bb3x"
server_key: "5ad806181bdc47836b414980b1fada31"
server_key_iv: "ba2717cb6360c4b2cfa9adc4b8c4b242"
server_key_version: "encryptionkey@0"
I20260923 15:06:44.922417 12765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:44.925717 12765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:44.925935 12765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:44.926057 12765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:44.930022 12765 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:42769
I20260923 15:06:44.930112 12831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:42769 every 8 connection(s)
I20260923 15:06:44.930457 12765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:44.931190 12832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:44.933951 12832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: Bootstrap starting.
I20260923 15:06:44.934598 12832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:44.934902 12832 log.cc:824] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:44.935716 12832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: No bootstrap required, opened a new log
I20260923 15:06:44.937522 12832 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfdc4793ac24daf9eca43b251baa125" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42769 } }
I20260923 15:06:44.937708 12832 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:44.937759 12832 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cfdc4793ac24daf9eca43b251baa125, State: Initialized, Role: FOLLOWER
I20260923 15:06:44.937950 12832 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfdc4793ac24daf9eca43b251baa125" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42769 } }
I20260923 15:06:44.938061 12832 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:44.938112 12832 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:44.938166 12832 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:44.938493 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12765
I20260923 15:06:44.938584 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:44.938791 8169 external_mini_cluster.cc:1428] Setting key 70f22c3231f66da9416b63aa9bd0f01b
I20260923 15:06:44.938973 12832 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfdc4793ac24daf9eca43b251baa125" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42769 } }
I20260923 15:06:44.939098 12832 leader_election.cc:302] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cfdc4793ac24daf9eca43b251baa125; no voters:
I20260923 15:06:44.939330 12832 leader_election.cc:288] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:44.939450 12835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:44.939648 12835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [term 1 LEADER]: Becoming Leader. State: Replica: 9cfdc4793ac24daf9eca43b251baa125, State: Running, Role: LEADER
I20260923 15:06:44.939625 12832 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:44.939808 12835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfdc4793ac24daf9eca43b251baa125" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42769 } }
I20260923 15:06:44.941450 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:44.939301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53374 (local address 127.7.250.126:42769)
0923 15:06:44.939693 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:44.939704 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:44.939739 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:06:44.939859 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:44.939867 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:44.939916 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:44.940079 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:44.940912 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:44.940916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:44.940917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:44.940923 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:44.940974 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:44.940992 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:44.941069 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:44.941162 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 15:06:44.941223 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":80,"threads_started":1}
I20260923 15:06:44.941550 12838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cfdc4793ac24daf9eca43b251baa125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfdc4793ac24daf9eca43b251baa125" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42769 } } }
I20260923 15:06:44.941782 12838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:44.941911 12851 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:44.941970 12851 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:44.942068 12853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:44.941429 12839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cfdc4793ac24daf9eca43b251baa125. Latest consensus state: current_term: 1 leader_uuid: "9cfdc4793ac24daf9eca43b251baa125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cfdc4793ac24daf9eca43b251baa125" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42769 } } }
I20260923 15:06:44.942178 12839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:44.942287 12853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:44.943874 12853 catalog_manager.cc:1347] Generated new cluster ID: 3a8abe369fac409c8f113e7aa47c4758
I20260923 15:06:44.943930 12853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:44.966732 12853 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:44.966934 12853 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:44.971150 12853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: Generated new TSK 0
I20260923 15:06:44.971333 12853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:44.975739 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42769
--builtin_ntp_servers=127.7.250.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 15:06:45.057742 12856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:45.057984 12856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:45.058023 12856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:45.059226 12856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:45.059299 12856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:45.059350 12856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:45.060703 12856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:45.060976 12856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:45.061281 12856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:45.067193 12861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.067226 12862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.067543 12864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:45.067544 12856 server_base.cc:1034] running on GCE node
I20260923 15:06:45.067834 12856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:45.068117 12856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:45.069303 12856 hybrid_clock.cc:648] HybridClock initialized: now 1790176005069280 us; error 33 us; skew 500 ppm
I20260923 15:06:45.070693 12856 webserver.cc:460] Webserver started at http://127.7.250.65:37191/ using document root <none> and password file <none>
I20260923 15:06:45.070986 12856 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:45.071062 12856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:45.071210 12856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:45.072321 12856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "4dcff5aee7684fa39e9acc67c9f30f7e"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "9a5bf26d88f41761501eb26873af0ac3"
server_key_iv: "c3b98f9f3138c5841cd661641488ff17"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.072697 12856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "4dcff5aee7684fa39e9acc67c9f30f7e"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "9a5bf26d88f41761501eb26873af0ac3"
server_key_iv: "c3b98f9f3138c5841cd661641488ff17"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.074064 12856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:06:45.075102 12870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.075318 12856 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:45.075445 12856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "4dcff5aee7684fa39e9acc67c9f30f7e"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "9a5bf26d88f41761501eb26873af0ac3"
server_key_iv: "c3b98f9f3138c5841cd661641488ff17"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.075546 12856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:45.079340 12856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:45.079645 12856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:45.079804 12856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:45.080156 12856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:45.080567 12856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:45.080636 12856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.080677 12856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:45.080698 12856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.086483 12856 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:36375
I20260923 15:06:45.086542 12983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:36375 every 8 connection(s)
I20260923 15:06:45.086948 12856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:45.090178 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.087717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:39349 (local address 127.7.250.126:42769)
0923 15:06:45.087779 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.087782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.088035 (+ 253us) server_negotiation.cc:407] Connection header received
0923 15:06:45.088564 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.088564 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.088572 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.088626 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.089861 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.089863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.089863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.089869 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.089888 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.089893 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.089982 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.090065 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.090081 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:06:45.090427 12986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.087709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42769 (local address 127.7.250.65:39349)
0923 15:06:45.087903 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.087942 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.088439 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.088713 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.088717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.088725 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.089692 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.089692 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.089734 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.089769 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.089983 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.089990 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.089994 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.090078 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.090150 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260923 15:06:45.090873 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12856
I20260923 15:06:45.091023 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:45.091214 8169 external_mini_cluster.cc:1428] Setting key b071d847a2de3d4b7a349842598520e9
I20260923 15:06:45.092226 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42769
--builtin_ntp_servers=127.7.250.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 15:06:45.094340 12984 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42769
I20260923 15:06:45.094447 12984 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:45.094611 12984 heartbeater.cc:511] Master 127.7.250.126:42769 requested a full tablet report, sending...
I20260923 15:06:45.095182 12796 ts_manager.cc:194] Registered new tserver with Master: 4dcff5aee7684fa39e9acc67c9f30f7e (127.7.250.65:36375)
W20260923 15:06:45.177651 12987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:45.177965 12987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:45.177994 12987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:45.179387 12987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:45.179431 12987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:45.179466 12987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:45.180832 12987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:45.181087 12987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:45.181393 12987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:45.186676 12995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.186708 12993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.186684 12992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:45.187093 12987 server_base.cc:1034] running on GCE node
I20260923 15:06:45.187309 12987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:45.187562 12987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:45.188750 12987 hybrid_clock.cc:648] HybridClock initialized: now 1790176005188743 us; error 38 us; skew 500 ppm
I20260923 15:06:45.190056 12987 webserver.cc:460] Webserver started at http://127.7.250.66:34945/ using document root <none> and password file <none>
I20260923 15:06:45.190382 12987 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:45.190527 12987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:45.190707 12987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:45.191856 12987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "57f04df14fc04b45bef12754e2b646f9"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "dc755f4ab754db6091253b608b54889b"
server_key_iv: "7a0d51eaed00103c7b76520b0d34b298"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.192271 12987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "57f04df14fc04b45bef12754e2b646f9"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "dc755f4ab754db6091253b608b54889b"
server_key_iv: "7a0d51eaed00103c7b76520b0d34b298"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.193717 12987 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:45.194602 13001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.194823 12987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:45.194944 12987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "57f04df14fc04b45bef12754e2b646f9"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "dc755f4ab754db6091253b608b54889b"
server_key_iv: "7a0d51eaed00103c7b76520b0d34b298"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.195026 12987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:45.198894 12987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:45.199189 12987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:45.199342 12987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:45.199564 12987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:45.199890 12987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:45.199957 12987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.200004 12987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:45.200034 12987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.206112 12987 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:41033
I20260923 15:06:45.206184 13114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:41033 every 8 connection(s)
I20260923 15:06:45.206585 12987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:06:45.207756 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 12987
I20260923 15:06:45.207876 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:45.208076 8169 external_mini_cluster.cc:1428] Setting key f65f75609d7ef14abb0f114aa17ea2b1
I20260923 15:06:45.209273 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42769
--builtin_ntp_servers=127.7.250.84:38505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 15:06:45.210319 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.207410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:57391 (local address 127.7.250.126:42769)
0923 15:06:45.207473 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.207475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.207669 (+ 194us) server_negotiation.cc:407] Connection header received
0923 15:06:45.208421 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.208423 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.208430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.208480 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.210019 (+ 1539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.210020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.210021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.210025 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.210039 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.210043 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.210093 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.210199 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.210211 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 15:06:45.210556 13117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.207330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42769 (local address 127.7.250.66:57391)
0923 15:06:45.207528 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.207573 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.208296 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.208555 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.208560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.208571 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.209826 (+ 1255us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.209827 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.209871 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.209912 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.210118 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.210122 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.210126 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.210205 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.210296 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260923 15:06:45.211392 13115 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42769
I20260923 15:06:45.211513 13115 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:45.211721 13115 heartbeater.cc:511] Master 127.7.250.126:42769 requested a full tablet report, sending...
I20260923 15:06:45.212141 12796 ts_manager.cc:194] Registered new tserver with Master: 57f04df14fc04b45bef12754e2b646f9 (127.7.250.66:41033)
W20260923 15:06:45.291874 13118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:45.292124 13118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:45.292162 13118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:45.293367 13118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:45.293447 13118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:45.293493 13118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:45.294862 13118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:45.295140 13118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:45.295454 13118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:45.300998 13126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.300985 13124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:45.301018 13118 server_base.cc:1034] running on GCE node
W20260923 15:06:45.300985 13123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:45.301643 13118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:45.301964 13118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:45.303124 13118 hybrid_clock.cc:648] HybridClock initialized: now 1790176005303106 us; error 38 us; skew 500 ppm
I20260923 15:06:45.304515 13118 webserver.cc:460] Webserver started at http://127.7.250.67:39691/ using document root <none> and password file <none>
I20260923 15:06:45.304829 13118 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:45.304906 13118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:45.305071 13118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:45.306169 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "56e771b85d9541e09a311f0b461a61c6"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "5f1870fa678be1d5b8a6812bb859b2db"
server_key_iv: "d42c05c3770713e5b91b2e467de5956f"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.306602 13118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "56e771b85d9541e09a311f0b461a61c6"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "5f1870fa678be1d5b8a6812bb859b2db"
server_key_iv: "d42c05c3770713e5b91b2e467de5956f"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.307874 13118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:45.308683 13132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.308897 13118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:45.309002 13118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "56e771b85d9541e09a311f0b461a61c6"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "5f1870fa678be1d5b8a6812bb859b2db"
server_key_iv: "d42c05c3770713e5b91b2e467de5956f"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.309087 13118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:45.312593 13118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:45.312871 13118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:45.313015 13118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:45.313230 13118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:45.313567 13118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:45.313624 13118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.313665 13118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:45.313692 13118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.319759 13118 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:36455
I20260923 15:06:45.319799 13245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:36455 every 8 connection(s)
I20260923 15:06:45.320163 13118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:45.323295 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.320969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:33907 (local address 127.7.250.126:42769)
0923 15:06:45.321011 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.321014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.321242 (+ 228us) server_negotiation.cc:407] Connection header received
0923 15:06:45.321718 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.321720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.321727 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.321798 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.323067 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.323068 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.323068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.323072 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.323085 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.323090 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.323154 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.323225 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.323238 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:06:45.323506 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.320939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42769 (local address 127.7.250.67:33907)
0923 15:06:45.321114 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.321149 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.321602 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.321852 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.321855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.321862 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.322894 (+ 1032us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.322894 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.322923 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.322960 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.323153 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.323155 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.323157 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.323216 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.323265 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260923 15:06:45.324055 13246 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42769
I20260923 15:06:45.324150 13246 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:45.324263 13246 heartbeater.cc:511] Master 127.7.250.126:42769 requested a full tablet report, sending...
I20260923 15:06:45.324611 12796 ts_manager.cc:194] Registered new tserver with Master: 56e771b85d9541e09a311f0b461a61c6 (127.7.250.67:36455)
I20260923 15:06:45.325106 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13118
I20260923 15:06:45.325212 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:45.325377 8169 external_mini_cluster.cc:1428] Setting key 75325ad04da1cbff928cab01927398f1
I20260923 15:06:45.326640 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:06:45.329322 12837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.328083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.7.250.126:42769)
0923 15:06:45.328142 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.328144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.328387 (+ 243us) server_negotiation.cc:407] Connection header received
0923 15:06:45.328416 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.328418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.328423 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.328480 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.329116 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.329117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.329118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.329122 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.329136 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.329140 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.329196 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.329258 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.329270 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:06:45.331115 12796 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:45.332661 12796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:45.337209 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.335231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:36375 (local address 127.0.0.1:52288)
0923 15:06:45.335501 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.335532 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.335595 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.336013 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.336020 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.336036 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.336720 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.336721 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.336725 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.336735 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.337040 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.337047 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.337050 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.337113 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.337127 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260923 15:06:45.337248 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.335684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.7.250.66:41033)
0923 15:06:45.335829 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.335835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.335846 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:45.335874 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.335876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.335888 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.335972 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.336930 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.336934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.336934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.336937 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.336992 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.337007 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.337084 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.337141 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.337156 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260923 15:06:45.337210 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.335236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41033 (local address 127.0.0.1:53144)
0923 15:06:45.335694 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.335703 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.335734 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.336034 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.336035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.336037 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.336664 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.336665 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.336682 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.336708 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.337081 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.337082 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.337083 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.337132 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.337142 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":115,"threads_started":1}
I20260923 15:06:45.337535 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.335506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52288 (local address 127.7.250.65:36375)
0923 15:06:45.335880 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.335884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.335892 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:45.335928 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.335930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.335938 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.336011 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.336886 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.336895 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.336895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.336900 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.336936 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.336962 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.337017 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.337428 (+ 411us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.337446 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":76,"threads_started":1}
I20260923 15:06:45.338003 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.335231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:36455 (local address 127.0.0.1:49418)
0923 15:06:45.335413 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.335532 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.335593 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.336548 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.336549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.336552 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.336902 (+ 350us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.336903 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.336908 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.336916 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.337153 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.337154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.337155 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.337186 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.337199 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":64,"threads_started":1}
I20260923 15:06:45.338056 13049 tablet_service.cc:1467] Processing CreateTablet for tablet 10cf7fc1efc147ce8b2343458f198b7d (DEFAULT_TABLE table=test-table [id=0d83dbb085d447d1b5d8971457eedc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:45.338312 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.335511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.7.250.67:36455)
0923 15:06:45.336346 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.336368 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.336379 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:45.336415 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.336418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.336428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.336509 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.337004 (+ 495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.337010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.337010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.337015 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.337046 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.337067 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.337249 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.338185 (+ 936us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.338202 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":798,"thread_start_us":81,"threads_started":1}
I20260923 15:06:45.338505 13049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10cf7fc1efc147ce8b2343458f198b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:45.338949 13180 tablet_service.cc:1467] Processing CreateTablet for tablet 10cf7fc1efc147ce8b2343458f198b7d (DEFAULT_TABLE table=test-table [id=0d83dbb085d447d1b5d8971457eedc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:45.339275 13180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10cf7fc1efc147ce8b2343458f198b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:45.341183 13260 tablet_bootstrap.cc:492] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Bootstrap starting.
I20260923 15:06:45.341632 12918 tablet_service.cc:1467] Processing CreateTablet for tablet 10cf7fc1efc147ce8b2343458f198b7d (DEFAULT_TABLE table=test-table [id=0d83dbb085d447d1b5d8971457eedc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:45.341879 13260 tablet_bootstrap.cc:654] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:45.341958 12918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10cf7fc1efc147ce8b2343458f198b7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:45.342212 13260 log.cc:824] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:45.342425 13261 tablet_bootstrap.cc:492] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Bootstrap starting.
I20260923 15:06:45.342972 13260 tablet_bootstrap.cc:492] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: No bootstrap required, opened a new log
I20260923 15:06:45.343045 13260 ts_tablet_manager.cc:1397] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:45.343133 13261 tablet_bootstrap.cc:654] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:45.343422 13261 log.cc:824] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:45.344108 13261 tablet_bootstrap.cc:492] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: No bootstrap required, opened a new log
I20260923 15:06:45.344194 13261 ts_tablet_manager.cc:1397] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:45.345615 13264 tablet_bootstrap.cc:492] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: Bootstrap starting.
I20260923 15:06:45.345726 13260 raft_consensus.cc:348] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.345906 13260 raft_consensus.cc:374] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:45.345956 13260 raft_consensus.cc:729] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f04df14fc04b45bef12754e2b646f9, State: Initialized, Role: FOLLOWER
I20260923 15:06:45.346083 13260 consensus_queue.cc:260] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.346381 13260 ts_tablet_manager.cc:1428] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:06:45.346383 13264 tablet_bootstrap.cc:654] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:45.346513 13115 heartbeater.cc:503] Master 127.7.250.126:42769 was elected leader, sending a full tablet report...
I20260923 15:06:45.346684 13264 log.cc:824] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:45.346606 13261 raft_consensus.cc:348] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.346755 13261 raft_consensus.cc:374] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:45.346784 13261 raft_consensus.cc:729] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e771b85d9541e09a311f0b461a61c6, State: Initialized, Role: FOLLOWER
I20260923 15:06:45.346899 13261 consensus_queue.cc:260] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.347162 13261 ts_tablet_manager.cc:1428] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:45.347201 13246 heartbeater.cc:503] Master 127.7.250.126:42769 was elected leader, sending a full tablet report...
I20260923 15:06:45.347782 13264 tablet_bootstrap.cc:492] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: No bootstrap required, opened a new log
I20260923 15:06:45.347883 13264 ts_tablet_manager.cc:1397] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:06:45.349941 13264 raft_consensus.cc:348] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.350061 13264 raft_consensus.cc:374] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:45.350081 13264 raft_consensus.cc:729] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dcff5aee7684fa39e9acc67c9f30f7e, State: Initialized, Role: FOLLOWER
I20260923 15:06:45.350201 13264 consensus_queue.cc:260] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.350484 13264 ts_tablet_manager.cc:1428] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:06:45.350564 12984 heartbeater.cc:503] Master 127.7.250.126:42769 was elected leader, sending a full tablet report...
W20260923 15:06:45.457172 13116 tablet.cc:2369] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:45.570724 13247 tablet.cc:2369] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:45.588099 12985 tablet.cc:2369] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:45.610221 13268 raft_consensus.cc:482] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:45.610471 13268 raft_consensus.cc:504] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.610836 13268 leader_election.cc:288] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56e771b85d9541e09a311f0b461a61c6 (127.7.250.67:36455), 57f04df14fc04b45bef12754e2b646f9 (127.7.250.66:41033)
I20260923 15:06:45.612684 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.611168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:52117 (local address 127.7.250.66:41033)
0923 15:06:45.611224 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.611226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.611267 (+ 41us) server_negotiation.cc:407] Connection header received
0923 15:06:45.611374 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.611375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.611381 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.611469 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.612291 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.612292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.612293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.612299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.612378 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.612385 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.612465 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.612580 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.612598 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:06:45.612677 13258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.611242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:49307 (local address 127.7.250.67:36455)
0923 15:06:45.611289 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.611291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.611305 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:06:45.611365 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.611366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.611373 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.611469 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.612404 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.612406 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.612406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.612412 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.612439 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.612445 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.612519 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.612578 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.612596 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:06:45.612702 13270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.611063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41033 (local address 127.7.250.65:52117)
0923 15:06:45.611193 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.611205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.611287 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.611516 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.611517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.611532 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.612145 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.612145 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.612151 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.612181 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.612500 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.612501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.612502 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.612564 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.612591 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":14,"thread_start_us":54,"threads_started":1}
I20260923 15:06:45.612702 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.611063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:36455 (local address 127.7.250.65:49307)
0923 15:06:45.611193 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.611205 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.611257 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.611501 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.611503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.611540 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:45.612215 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:06:45.612216 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:45.612221 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:06:45.612239 (+ 18us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:45.612516 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:45.612517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:45.612517 (+ 0us) client_negotiation.cc:769] Sending connection context
0923 15:06:45.612564 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:06:45.612598 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":17,"thread_start_us":68,"threads_started":1}
I20260923 15:06:45.613168 13200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cf7fc1efc147ce8b2343458f198b7d" candidate_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e771b85d9541e09a311f0b461a61c6" is_pre_election: true
I20260923 15:06:45.613235 13069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cf7fc1efc147ce8b2343458f198b7d" candidate_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f04df14fc04b45bef12754e2b646f9" is_pre_election: true
I20260923 15:06:45.613328 13200 raft_consensus.cc:2454] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dcff5aee7684fa39e9acc67c9f30f7e in term 0.
I20260923 15:06:45.613395 13069 raft_consensus.cc:2454] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4dcff5aee7684fa39e9acc67c9f30f7e in term 0.
I20260923 15:06:45.613574 12872 leader_election.cc:302] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dcff5aee7684fa39e9acc67c9f30f7e, 56e771b85d9541e09a311f0b461a61c6; no voters:
I20260923 15:06:45.613713 13268 raft_consensus.cc:2781] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:45.613781 13268 raft_consensus.cc:482] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:45.613808 13268 raft_consensus.cc:3037] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:45.614539 13268 raft_consensus.cc:504] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.614707 13268 leader_election.cc:288] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [CANDIDATE]: Term 1 election: Requested vote from peers 56e771b85d9541e09a311f0b461a61c6 (127.7.250.67:36455), 57f04df14fc04b45bef12754e2b646f9 (127.7.250.66:41033)
I20260923 15:06:45.614889 13200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cf7fc1efc147ce8b2343458f198b7d" candidate_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e771b85d9541e09a311f0b461a61c6"
I20260923 15:06:45.614899 13069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10cf7fc1efc147ce8b2343458f198b7d" candidate_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57f04df14fc04b45bef12754e2b646f9"
I20260923 15:06:45.614989 13069 raft_consensus.cc:3037] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:45.614993 13200 raft_consensus.cc:3037] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:45.615658 13069 raft_consensus.cc:2454] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dcff5aee7684fa39e9acc67c9f30f7e in term 1.
I20260923 15:06:45.615784 13200 raft_consensus.cc:2454] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4dcff5aee7684fa39e9acc67c9f30f7e in term 1.
I20260923 15:06:45.615844 12873 leader_election.cc:302] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dcff5aee7684fa39e9acc67c9f30f7e, 57f04df14fc04b45bef12754e2b646f9; no voters:
I20260923 15:06:45.615926 13268 raft_consensus.cc:2781] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:45.616081 13268 raft_consensus.cc:686] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 1 LEADER]: Becoming Leader. State: Replica: 4dcff5aee7684fa39e9acc67c9f30f7e, State: Running, Role: LEADER
I20260923 15:06:45.616178 13268 consensus_queue.cc:237] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } }
I20260923 15:06:45.617017 12796 catalog_manager.cc:5495] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dcff5aee7684fa39e9acc67c9f30f7e (127.7.250.65). New cstate: current_term: 1 leader_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4dcff5aee7684fa39e9acc67c9f30f7e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:45.623454 13257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.622352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52302 (local address 127.7.250.65:36375)
0923 15:06:45.622417 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.622478 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.622486 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:45.622520 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.622521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.622539 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.622606 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.623219 (+ 613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.623220 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.623221 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.623226 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.623242 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.623246 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.623294 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.623360 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.623374 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:06:45.625181 13069 raft_consensus.cc:1261] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 1 FOLLOWER]: Refusing update from remote peer 4dcff5aee7684fa39e9acc67c9f30f7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:45.625181 13200 raft_consensus.cc:1261] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 1 FOLLOWER]: Refusing update from remote peer 4dcff5aee7684fa39e9acc67c9f30f7e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:45.625550 13271 consensus_queue.cc:1035] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e771b85d9541e09a311f0b461a61c6" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:45.625691 13268 consensus_queue.cc:1035] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "57f04df14fc04b45bef12754e2b646f9" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:45.628077 13274 mvcc.cc:205] Tried to move back new op lower bound from 7332560919038058496 to 7332560919004672000. Current Snapshot: MvccSnapshot[applied={T|T < 7332560919038058496}]
I20260923 15:06:45.630355 12795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:45.630451 12795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:45.631218 12795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cfdc4793ac24daf9eca43b251baa125: Sending DeleteTablet for 3 replicas of tablet 10cf7fc1efc147ce8b2343458f198b7d
I20260923 15:06:45.631548 13049 tablet_service.cc:1514] Processing DeleteTablet for tablet 10cf7fc1efc147ce8b2343458f198b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:45 UTC) from {username='slave'} at 127.0.0.1:53144
I20260923 15:06:45.631548 12918 tablet_service.cc:1514] Processing DeleteTablet for tablet 10cf7fc1efc147ce8b2343458f198b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:45 UTC) from {username='slave'} at 127.0.0.1:52288
I20260923 15:06:45.631781 13285 tablet_replica.cc:330] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e: stopping tablet replica
I20260923 15:06:45.631778 13284 tablet_replica.cc:330] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: stopping tablet replica
I20260923 15:06:45.631878 13284 raft_consensus.cc:2229] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:45.631883 13285 raft_consensus.cc:2229] T 10cf7fc1efc147ce8b2343458f198b7d P 4dcff5aee7684fa39e9acc67c9f30f7e [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:45.632004 13284 raft_consensus.cc:2258] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:45.632110 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12856
I20260923 15:06:45.632429 13284 ts_tablet_manager.cc:1905] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:45.632810 13180 tablet_service.cc:1514] Processing DeleteTablet for tablet 10cf7fc1efc147ce8b2343458f198b7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:45 UTC) from {username='slave'} at 127.0.0.1:49418
I20260923 15:06:45.633066 13286 tablet_replica.cc:330] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: stopping tablet replica
I20260923 15:06:45.633165 13286 raft_consensus.cc:2229] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:45.633266 13286 raft_consensus.cc:2258] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:45.633702 13286 ts_tablet_manager.cc:1905] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:45.634473 13284 ts_tablet_manager.cc:1918] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:45.634531 13284 log.cc:1196] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/10cf7fc1efc147ce8b2343458f198b7d
I20260923 15:06:45.634720 13284 ts_tablet_manager.cc:1939] T 10cf7fc1efc147ce8b2343458f198b7d P 57f04df14fc04b45bef12754e2b646f9: Deleting consensus metadata
I20260923 15:06:45.635077 12782 catalog_manager.cc:4841] TS 57f04df14fc04b45bef12754e2b646f9 (127.7.250.66:41033): tablet 10cf7fc1efc147ce8b2343458f198b7d (table test-table [id=0d83dbb085d447d1b5d8971457eedc14]) successfully deleted
I20260923 15:06:45.635664 13286 ts_tablet_manager.cc:1918] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:45.635731 13286 log.cc:1196] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/10cf7fc1efc147ce8b2343458f198b7d
I20260923 15:06:45.635991 13286 ts_tablet_manager.cc:1939] T 10cf7fc1efc147ce8b2343458f198b7d P 56e771b85d9541e09a311f0b461a61c6: Deleting consensus metadata
I20260923 15:06:45.636503 12781 catalog_manager.cc:4841] TS 56e771b85d9541e09a311f0b461a61c6 (127.7.250.67:36455): tablet 10cf7fc1efc147ce8b2343458f198b7d (table test-table [id=0d83dbb085d447d1b5d8971457eedc14]) successfully deleted
W20260923 15:06:45.640179 12783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:36375 (error 108)
I20260923 15:06:45.640743 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12987
I20260923 15:06:45.640779 13255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.640642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:36375 (local address 127.0.0.1:52308)
0923 15:06:45.640700 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.640749 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:36375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 15:06:45.645483 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13118
W20260923 15:06:45.645505 12783 catalog_manager.cc:4584] TS 4dcff5aee7684fa39e9acc67c9f30f7e (127.7.250.65:36375): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10cf7fc1efc147ce8b2343458f198b7d: Network error: Client connection negotiation failed: client connection to 127.7.250.65:36375: connect: Connection refused (error 111)
I20260923 15:06:45.650005 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 12765
2026-09-23T15:06:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (852 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 15:06:45.669466 8169 mini_oidc.cc:152] Starting JWKS server
I20260923 15:06:45.670789 8169 webserver.cc:460] Webserver started at https://127.0.0.1:40963/ using document root <none> and password file <none>
I20260923 15:06:45.670866 8169 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 15:06:45.671049 8169 webserver.cc:460] Webserver started at http://127.0.0.1:34761/ using document root <none> and password file <none>
2026-09-23T15:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:45Z Disabled control of system clock
I20260923 15:06:45.679222 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41019
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:34691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41019
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40963/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/testchainca.cert with env {}
W20260923 15:06:45.760705 13293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:45.760924 13293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:45.760947 13293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:45.762063 13293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:45.762097 13293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:45.762110 13293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:45.762122 13293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:45.762135 13293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:06:45.762147 13293 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:06:45.762158 13293 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40963/jwks.json
I20260923 15:06:45.763830 13293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:34691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40963/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:45.764035 13293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:45.764209 13293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:45.769305 13301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.769305 13299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:45.769517 13298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:45.769439 13293 server_base.cc:1034] running on GCE node
I20260923 15:06:45.769791 13293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:45.770051 13293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:45.771203 13293 hybrid_clock.cc:648] HybridClock initialized: now 1790176005771172 us; error 48 us; skew 500 ppm
I20260923 15:06:45.775933 13293 webserver.cc:460] Webserver started at http://127.7.250.126:45869/ using document root <none> and password file <none>
I20260923 15:06:45.776218 13293 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:45.776268 13293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:45.776404 13293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:45.777549 13293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "f014b40b55cdd17c414efcf9074cb456"
server_key_iv: "5451c47f8ea2d505dc6da4f77892ffbd"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.777915 13293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "f014b40b55cdd17c414efcf9074cb456"
server_key_iv: "5451c47f8ea2d505dc6da4f77892ffbd"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.779388 13293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:45.780186 13307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:45.780591 13293 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:45.780711 13293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434"
format_stamp: "Formatted at 2026-09-23 15:06:45 on dist-test-slave-bb3x"
server_key: "f014b40b55cdd17c414efcf9074cb456"
server_key_iv: "5451c47f8ea2d505dc6da4f77892ffbd"
server_key_version: "encryptionkey@0"
I20260923 15:06:45.780791 13293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:45.790762 13293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:45.791091 13293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:45.791244 13293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:45.795763 13293 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41019
I20260923 15:06:45.795822 13359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41019 every 8 connection(s)
I20260923 15:06:45.796198 13293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:45.797295 13360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:45.800027 13360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434: Bootstrap starting.
I20260923 15:06:45.800660 13360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:45.800947 13360 log.cc:824] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:45.801642 13360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434: No bootstrap required, opened a new log
I20260923 15:06:45.803583 13360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41019 } }
I20260923 15:06:45.803747 13360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:45.803777 13360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5aa0fd9d74ad4a53b797a1a0bbac5434, State: Initialized, Role: FOLLOWER
I20260923 15:06:45.803977 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13293
I20260923 15:06:45.803953 13360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41019 } }
I20260923 15:06:45.804040 13360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:45.804083 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:45.804069 13360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:45.804143 13360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:45.804287 8169 external_mini_cluster.cc:1428] Setting key da3e9e217fe7fb566b64d6d32d669e7c
I20260923 15:06:45.805063 13360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41019 } }
I20260923 15:06:45.805192 13360 leader_election.cc:302] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5aa0fd9d74ad4a53b797a1a0bbac5434; no voters:
I20260923 15:06:45.805370 13360 leader_election.cc:288] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:45.805400 13365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:45.805557 13365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [term 1 LEADER]: Becoming Leader. State: Replica: 5aa0fd9d74ad4a53b797a1a0bbac5434, State: Running, Role: LEADER
I20260923 15:06:45.805660 13360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:45.805650 13365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41019 } }
I20260923 15:06:45.806118 13366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5aa0fd9d74ad4a53b797a1a0bbac5434. Latest consensus state: current_term: 1 leader_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41019 } } }
I20260923 15:06:45.806226 13366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:45.806212 13367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5aa0fd9d74ad4a53b797a1a0bbac5434" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41019 } } }
I20260923 15:06:45.806927 13367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:45.807066 13374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:45.807541 13374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:45.809306 13374 catalog_manager.cc:1347] Generated new cluster ID: 6524910a50064b2bad732998b98f0132
I20260923 15:06:45.809376 13374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:45.811600 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.804899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.7.250.126:41019)
0923 15:06:45.805175 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.805186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.805227 (+ 41us) server_negotiation.cc:407] Connection header received
0923 15:06:45.805395 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.805403 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.805488 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.805972 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:45.806112 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:45.807792 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:45.809040 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:45.809694 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:45.809860 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:45.810777 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:45.810781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:45.810781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:45.810984 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:45.811021 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:45.811033 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:45.811126 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:45.811215 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 15:06:45.811412 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":104,"threads_started":1}
I20260923 15:06:45.815052 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:45.815801 13374 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:45.816378 13374 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:45.823724 13374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5aa0fd9d74ad4a53b797a1a0bbac5434: Generated new TSK 0
I20260923 15:06:45.823948 13374 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 15:06:45.856611 13390 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 15:06:45.856779 13390 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 15:06:45.856983 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:45.833402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34644 (local address 127.7.250.126:41019)
0923 15:06:45.833449 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:45.833452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:45.833468 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:06:45.833570 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:45.833571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:45.833582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:45.833649 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:06:45.833835 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:45.834539 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:45.835773 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:45.835973 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:45.836035 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:45.836270 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:06:45.856586 (+ 20316us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 15:06:45.856913 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:45.857043 13364 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 15:06:45.857228 13389 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:45.833226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41019 (local address 127.0.0.1:34644)
0923 15:06:45.833414 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:45.833425 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:45.833479 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:45.833685 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:45.833685 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:45.833687 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:06:45.833757 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:45.833760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:45.834626 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:45.834627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:45.835680 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:45.835684 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:45.836064 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:45.836065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:45.836134 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:45.836158 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:06:45.856746 (+ 20588us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 15:06:45.856873 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.250.126:41019: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.250.126:41019: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40963/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 15:06:45.857846 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13293
2026-09-23T15:06:45Z chronyd exiting
I20260923 15:06:45.893813 8169 test_util.cc:175] -----------------------------------------------
I20260923 15:06:45.893921 8169 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175983307587-8169-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 15:06:45.895191 8169 mini_oidc.cc:152] Starting JWKS server
I20260923 15:06:45.896042 8169 webserver.cc:460] Webserver started at https://127.0.0.1:43827/ using document root <none> and password file <none>
I20260923 15:06:45.896111 8169 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 15:06:45.896272 8169 webserver.cc:460] Webserver started at http://127.0.0.1:37845/ using document root <none> and password file <none>
2026-09-23T15:06:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:45Z Disabled control of system clock
I20260923 15:06:45.904711 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40819
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:35013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:40819
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43827/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/testchainca.cert with env {}
W20260923 15:06:45.993160 13396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:45.993391 13396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:45.993414 13396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:45.994611 13396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:45.994661 13396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:45.994674 13396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:45.994691 13396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:45.994704 13396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:06:45.994716 13396 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:06:45.994727 13396 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43827/jwks.json
I20260923 15:06:45.996279 13396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:35013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:40819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43827/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:45.996500 13396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:45.996671 13396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:46.002293 13404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:46.002314 13401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:46.002342 13396 server_base.cc:1034] running on GCE node
W20260923 15:06:46.002317 13402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:46.002871 13396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:46.003211 13396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:46.004412 13396 hybrid_clock.cc:648] HybridClock initialized: now 1790176006004388 us; error 52 us; skew 500 ppm
I20260923 15:06:46.006067 13396 webserver.cc:460] Webserver started at http://127.7.250.126:44129/ using document root <none> and password file <none>
I20260923 15:06:46.006512 13396 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:46.006604 13396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:46.006778 13396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:46.008044 13396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "c997ae490d944e15a55a46edf91d7448"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "217ccae3ad12bee42c928e84749ee1c8"
server_key_iv: "d3efbe83ac656da9dd70e0a83f63a8c7"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.008545 13396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "c997ae490d944e15a55a46edf91d7448"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "217ccae3ad12bee42c928e84749ee1c8"
server_key_iv: "d3efbe83ac656da9dd70e0a83f63a8c7"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.009914 13396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:46.010768 13410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:46.010944 13396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:46.011053 13396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "c997ae490d944e15a55a46edf91d7448"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "217ccae3ad12bee42c928e84749ee1c8"
server_key_iv: "d3efbe83ac656da9dd70e0a83f63a8c7"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.011140 13396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:46.031955 13396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:46.032375 13396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:46.032565 13396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:46.037176 13396 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:40819
I20260923 15:06:46.037216 13462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:40819 every 8 connection(s)
I20260923 15:06:46.037603 13396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:46.038697 13463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:46.039669 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13396
I20260923 15:06:46.039805 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:46.040009 8169 external_mini_cluster.cc:1428] Setting key 0b56e0c9873894ce06b8a4ae5eb4cbe2
I20260923 15:06:46.041602 13463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448: Bootstrap starting.
I20260923 15:06:46.042222 13463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:46.042529 13463 log.cc:824] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:46.043377 13463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448: No bootstrap required, opened a new log
I20260923 15:06:46.045740 13463 raft_consensus.cc:348] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997ae490d944e15a55a46edf91d7448" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40819 } }
I20260923 15:06:46.045953 13463 raft_consensus.cc:374] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:46.046020 13463 raft_consensus.cc:729] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c997ae490d944e15a55a46edf91d7448, State: Initialized, Role: FOLLOWER
I20260923 15:06:46.046195 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:46.040646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51808 (local address 127.7.250.126:40819)
0923 15:06:46.040835 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:46.040842 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:46.040869 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:06:46.041361 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:46.041368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:46.041432 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:46.041549 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:46.041826 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.043002 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.044117 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.044569 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.044664 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.045384 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:46.045388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:46.045388 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:46.045597 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:46.045630 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:46.045641 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:46.045693 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:46.045804 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 15:06:46.045900 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260923 15:06:46.046183 13463 consensus_queue.cc:260] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997ae490d944e15a55a46edf91d7448" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40819 } }
I20260923 15:06:46.046310 13463 raft_consensus.cc:388] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:46.046356 13463 raft_consensus.cc:482] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:46.046412 13463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:46.047168 13463 raft_consensus.cc:504] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997ae490d944e15a55a46edf91d7448" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40819 } }
I20260923 15:06:46.047273 13463 leader_election.cc:302] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c997ae490d944e15a55a46edf91d7448; no voters:
I20260923 15:06:46.047305 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:46.047435 13463 leader_election.cc:288] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:46.047465 13468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:46.047570 13468 raft_consensus.cc:686] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [term 1 LEADER]: Becoming Leader. State: Replica: c997ae490d944e15a55a46edf91d7448, State: Running, Role: LEADER
I20260923 15:06:46.047649 13468 consensus_queue.cc:237] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997ae490d944e15a55a46edf91d7448" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40819 } }
I20260923 15:06:46.047686 13463 sys_catalog.cc:564] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:46.047998 13468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c997ae490d944e15a55a46edf91d7448. Latest consensus state: current_term: 1 leader_uuid: "c997ae490d944e15a55a46edf91d7448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997ae490d944e15a55a46edf91d7448" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40819 } } }
I20260923 15:06:46.048086 13468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:46.048007 13469 sys_catalog.cc:455] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c997ae490d944e15a55a46edf91d7448" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c997ae490d944e15a55a46edf91d7448" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40819 } } }
I20260923 15:06:46.048398 13469 sys_catalog.cc:458] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:46.048588 13477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:46.049129 13477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:46.050743 13477 catalog_manager.cc:1347] Generated new cluster ID: 795f0bdb4652405091004b008980e452
I20260923 15:06:46.050807 13477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:46.070137 13477 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:46.070722 13477 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:46.078867 13477 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c997ae490d944e15a55a46edf91d7448: Generated new TSK 0
I20260923 15:06:46.079102 13477 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 15:06:46.106302 13493 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 15:06:46.106400 13493 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 15:06:46.106704 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:46.092071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.7.250.126:40819)
0923 15:06:46.092142 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:46.092145 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:46.092239 (+ 94us) server_negotiation.cc:407] Connection header received
0923 15:06:46.092317 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:46.092318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:46.092328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:46.092402 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:06:46.092558 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.093083 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.094228 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.094512 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.094572 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.094769 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:06:46.106529 (+ 11760us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 15:06:46.106652 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51824: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":44}
W20260923 15:06:46.106781 13466 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51824: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 15:06:46.106938 13492 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:46.092011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40819 (local address 127.0.0.1:51824)
0923 15:06:46.092164 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:46.092175 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:46.092241 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:46.092424 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:46.092425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:46.092427 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:06:46.092487 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:46.092489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.093185 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.093186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:46.094127 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:46.094130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.094595 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.094596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:46.094660 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.094667 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:06:46.106657 (+ 11990us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 15:06:46.106792 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.250.126:40819: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43827/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":105,"threads_started":1}
I20260923 15:06:46.107275 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13396
2026-09-23T15:06:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (250 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 15:06:46.144877 8169 mini_oidc.cc:152] Starting JWKS server
I20260923 15:06:46.145612 8169 webserver.cc:460] Webserver started at https://127.0.0.1:37503/ using document root <none> and password file <none>
I20260923 15:06:46.145654 8169 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 15:06:46.145810 8169 webserver.cc:460] Webserver started at http://127.0.0.1:33671/ using document root <none> and password file <none>
2026-09-23T15:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:46Z Disabled control of system clock
I20260923 15:06:46.153642 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44629
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:41699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:44629
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37503/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 15:06:46.234658 13499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:46.234946 13499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:46.234993 13499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:46.236222 13499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:46.236297 13499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:46.236316 13499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:46.236336 13499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:46.236369 13499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:06:46.236442 13499 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:06:46.236475 13499 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37503/jwks.json
I20260923 15:06:46.237924 13499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:44629
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37503/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:46.238219 13499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:46.238564 13499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:46.244105 13507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:46.244114 13505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:46.244143 13504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:46.244161 13499 server_base.cc:1034] running on GCE node
I20260923 15:06:46.244589 13499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:46.244899 13499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:46.246071 13499 hybrid_clock.cc:648] HybridClock initialized: now 1790176006246050 us; error 39 us; skew 500 ppm
I20260923 15:06:46.247476 13499 webserver.cc:460] Webserver started at http://127.7.250.126:33067/ using document root <none> and password file <none>
I20260923 15:06:46.247748 13499 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:46.247795 13499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:46.247991 13499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:46.249081 13499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "cb3fcf55c6fb1aeeb42441729ef15ddc"
server_key_iv: "a1757ff3058a8f7ddb2abc38b2198d67"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.249382 13499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "cb3fcf55c6fb1aeeb42441729ef15ddc"
server_key_iv: "a1757ff3058a8f7ddb2abc38b2198d67"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.250778 13499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:46.251569 13513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:46.251760 13499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:46.251873 13499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "cb3fcf55c6fb1aeeb42441729ef15ddc"
server_key_iv: "a1757ff3058a8f7ddb2abc38b2198d67"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.251965 13499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:46.269968 13499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:46.270335 13499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:46.270539 13499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:46.274780 13499 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:44629
I20260923 15:06:46.274806 13565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:44629 every 8 connection(s)
I20260923 15:06:46.275205 13499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:46.276186 13566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:46.278049 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13499
I20260923 15:06:46.278128 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:46.278335 8169 external_mini_cluster.cc:1428] Setting key e115e57fecd130c49e0e6b58b4db77f6
I20260923 15:06:46.278764 13566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b: Bootstrap starting.
I20260923 15:06:46.279402 13566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:46.279691 13566 log.cc:824] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:46.280352 13566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b: No bootstrap required, opened a new log
I20260923 15:06:46.282541 13566 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44629 } }
I20260923 15:06:46.282701 13566 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:46.282747 13566 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cbc1b48bdef4cbfa3a81c53a0e7e66b, State: Initialized, Role: FOLLOWER
I20260923 15:06:46.282889 13566 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44629 } }
I20260923 15:06:46.282991 13566 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:46.283026 13566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:46.283072 13566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:46.283741 13566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44629 } }
I20260923 15:06:46.283835 13566 leader_election.cc:302] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cbc1b48bdef4cbfa3a81c53a0e7e66b; no voters:
I20260923 15:06:46.284010 13566 leader_election.cc:288] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:46.284063 13571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:46.284211 13571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [term 1 LEADER]: Becoming Leader. State: Replica: 4cbc1b48bdef4cbfa3a81c53a0e7e66b, State: Running, Role: LEADER
I20260923 15:06:46.284274 13566 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:46.284309 13571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44629 } }
I20260923 15:06:46.284647 13571 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cbc1b48bdef4cbfa3a81c53a0e7e66b. Latest consensus state: current_term: 1 leader_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44629 } } }
I20260923 15:06:46.284660 13572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cbc1b48bdef4cbfa3a81c53a0e7e66b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44629 } } }
I20260923 15:06:46.284739 13571 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:46.284749 13572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:46.284967 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:46.278942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.7.250.126:44629)
0923 15:06:46.279154 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:46.279167 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:46.279194 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:06:46.279690 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:46.279698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:46.279793 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:46.279952 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:46.280173 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.281177 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.282442 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.282921 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.283059 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.283705 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:46.283709 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:46.283709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:46.283941 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:46.283975 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:46.283985 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:46.284085 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:46.284447 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 15:06:46.284555 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260923 15:06:46.285162 13578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:46.285631 13578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:46.285951 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:46.287217 13578 catalog_manager.cc:1347] Generated new cluster ID: 188bd9a8c9524de68f31e298229fb6c5
I20260923 15:06:46.287273 13578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:46.296355 13578 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:46.296928 13578 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:46.301159 13578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cbc1b48bdef4cbfa3a81c53a0e7e66b: Generated new TSK 0
I20260923 15:06:46.301349 13578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:46.315050 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:46.310940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.7.250.126:44629)
0923 15:06:46.310993 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:46.310995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:46.311084 (+ 89us) server_negotiation.cc:407] Connection header received
0923 15:06:46.311151 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:46.311152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:46.311162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:46.311221 (+ 59us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:06:46.311354 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.311889 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.313046 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.313548 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.313606 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.313808 (+ 202us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:06:46.314801 (+ 993us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37503/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 15:06:46.314965 (+ 164us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37503/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:46.315112 13569 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37503/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 15:06:46.315135 13595 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:46.310826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44629 (local address 127.0.0.1:54808)
0923 15:06:46.311000 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:46.311012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:46.311080 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:46.311222 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:46.311223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:46.311225 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:06:46.311293 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:46.311295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.311991 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.311992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:46.312947 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:46.312949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.313635 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.313637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:46.313705 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.313712 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:06:46.314936 (+ 1224us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37503/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 15:06:46.315028 (+ 92us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.250.126:44629: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37503/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260923 15:06:46.315660 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13499
E20260923 15:06:46.326195 13596 webserver.cc:533] Webserver: sslize failed:
E20260923 15:06:46.326319 13596 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T15:06:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (208 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T15:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:46Z Disabled control of system clock
I20260923 15:06:46.360741 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37633
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:45359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:37633
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/testchainca.cert with env {}
W20260923 15:06:46.440838 13600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:46.441099 13600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:46.441140 13600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:46.442358 13600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:46.442440 13600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:46.442461 13600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:46.442482 13600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:46.442512 13600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:06:46.442529 13600 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:06:46.442548 13600 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 15:06:46.444044 13600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:37633
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:46.444332 13600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:46.444557 13600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:46.450400 13600 server_base.cc:1034] running on GCE node
W20260923 15:06:46.450371 13605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:46.450420 13606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:46.450371 13608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:46.450810 13600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:46.451131 13600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:46.452291 13600 hybrid_clock.cc:648] HybridClock initialized: now 1790176006452257 us; error 51 us; skew 500 ppm
I20260923 15:06:46.453683 13600 webserver.cc:460] Webserver started at http://127.7.250.126:43605/ using document root <none> and password file <none>
I20260923 15:06:46.453918 13600 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:46.453962 13600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:46.454051 13600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:46.455143 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "6304b293ccdf4a86808c2cfc22fe70e4"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "45eae7cb473789e860fd54f3a83bdf7b"
server_key_iv: "354afa3ef235b378ef7213dfd2e4ab79"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.455438 13600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "6304b293ccdf4a86808c2cfc22fe70e4"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "45eae7cb473789e860fd54f3a83bdf7b"
server_key_iv: "354afa3ef235b378ef7213dfd2e4ab79"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.456687 13600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:06:46.457450 13614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:46.457662 13600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:46.457772 13600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "6304b293ccdf4a86808c2cfc22fe70e4"
format_stamp: "Formatted at 2026-09-23 15:06:46 on dist-test-slave-bb3x"
server_key: "45eae7cb473789e860fd54f3a83bdf7b"
server_key_iv: "354afa3ef235b378ef7213dfd2e4ab79"
server_key_version: "encryptionkey@0"
I20260923 15:06:46.457870 13600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:46.471889 13600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:46.472241 13600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:46.472402 13600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:46.476580 13600 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:37633
I20260923 15:06:46.476650 13666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:37633 every 8 connection(s)
I20260923 15:06:46.477017 13600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:46.477981 13667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:46.480360 13667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4: Bootstrap starting.
I20260923 15:06:46.480952 13667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:46.481245 13667 log.cc:824] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:46.482234 13667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4: No bootstrap required, opened a new log
I20260923 15:06:46.484110 13667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37633 } }
I20260923 15:06:46.484263 13667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:46.484289 13667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6304b293ccdf4a86808c2cfc22fe70e4, State: Initialized, Role: FOLLOWER
I20260923 15:06:46.484457 13667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37633 } }
I20260923 15:06:46.484535 13667 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:46.484581 13667 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:46.484651 13667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:46.485261 13667 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37633 } }
I20260923 15:06:46.485395 13667 leader_election.cc:302] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6304b293ccdf4a86808c2cfc22fe70e4; no voters:
I20260923 15:06:46.485591 13667 leader_election.cc:288] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:46.485606 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13600
I20260923 15:06:46.485652 13670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:46.485713 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:46.485808 13670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [term 1 LEADER]: Becoming Leader. State: Replica: 6304b293ccdf4a86808c2cfc22fe70e4, State: Running, Role: LEADER
I20260923 15:06:46.485884 13667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:46.485915 8169 external_mini_cluster.cc:1428] Setting key 6fc0cde16d1da3c24ad77ed98211f551
I20260923 15:06:46.485908 13670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37633 } }
I20260923 15:06:46.486281 13670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6304b293ccdf4a86808c2cfc22fe70e4. Latest consensus state: current_term: 1 leader_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37633 } } }
I20260923 15:06:46.486361 13670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:46.486768 13675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:46.487293 13675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:46.487782 13671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6304b293ccdf4a86808c2cfc22fe70e4" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37633 } } }
I20260923 15:06:46.487944 13671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:46.489584 13675 catalog_manager.cc:1347] Generated new cluster ID: 0aea4b3aa0b54eba803861f1b7b5de58
I20260923 15:06:46.489638 13675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:46.492324 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:46.486440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.7.250.126:37633)
0923 15:06:46.486818 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:46.486828 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:46.487065 (+ 237us) server_negotiation.cc:407] Connection header received
0923 15:06:46.487186 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:46.487193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:46.487263 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:46.487400 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:46.487724 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.489133 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.490278 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.490724 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.490898 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.491622 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:46.491626 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:46.491626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:46.491833 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:46.491866 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:46.491884 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:46.491972 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:46.492053 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 15:06:46.492141 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":67,"threads_started":1}
I20260923 15:06:46.493274 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:46.502394 13675 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:46.503090 13675 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:46.514282 13675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6304b293ccdf4a86808c2cfc22fe70e4: Generated new TSK 0
I20260923 15:06:46.514525 13675 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:46.600598 13677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:46.524451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38986 (local address 127.7.250.126:37633)
0923 15:06:46.524501 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:46.524503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:46.524587 (+ 84us) server_negotiation.cc:407] Connection header received
0923 15:06:46.524660 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:46.524661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:46.524670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:46.524731 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:06:46.524929 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.525429 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.526603 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.526786 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.526842 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.527033 (+ 191us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:06:46.600354 (+ 73321us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 15:06:46.600537 (+ 183us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38986: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 15:06:46.600689 13677 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38986: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 15:06:46.600811 13696 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:46.524332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37633 (local address 127.0.0.1:38986)
0923 15:06:46.524509 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:46.524520 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:46.524589 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:46.524755 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:46.524756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:46.524758 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:06:46.524829 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:46.524831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.525518 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.525519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:46.526522 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:46.526526 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:46.526868 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:46.526870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:46.526934 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:46.526941 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:06:46.600529 (+ 73588us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 15:06:46.600686 (+ 157us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.250.126:37633: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":122,"threads_started":1}
I20260923 15:06:46.601181 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13600
2026-09-23T15:06:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (265 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (99 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:46 dist-test-slave-bb3x krb5kdc[13708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:46 dist-test-slave-bb3x krb5kdc[13708](info): set up 2 sockets
Sep 23 15:06:46 dist-test-slave-bb3x krb5kdc[13708](info): commencing operation
krb5kdc: starting...
W20260923 15:06:47.917762 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260923 15:06:48.176471 8169 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.259s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260923 15:06:48.392625 8169 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.216s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260923 15:06:48.624562 8169 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.232s user 0.000s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/test-user.keytab.
W20260923 15:06:49.477140 8169 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.852s user 0.000s sys 0.002s
Sep 23 15:06:49 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:49.541900 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41933
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:49.622773 13721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:49.623042 13721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:49.623085 13721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:49.624276 13721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:49.624354 13721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:49.624378 13721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:49.624400 13721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:49.624428 13721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:49.625952 13721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:49.626246 13721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:49.626507 13721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:49.631757 13729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:49.631790 13726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:49.631757 13727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:49.632033 13721 server_base.cc:1034] running on GCE node
I20260923 15:06:49.632303 13721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:49.632833 13721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:49.634042 13721 hybrid_clock.cc:648] HybridClock initialized: now 1790176009634013 us; error 47 us; skew 500 ppm
Sep 23 15:06:49 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:49.636825 13721 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:06:49.637226 13721 webserver.cc:460] Webserver started at http://127.7.250.126:43011/ using document root <none> and password file <none>
I20260923 15:06:49.637537 13721 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:49.637581 13721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:49.637670 13721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:49.638754 13721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "fc6ee47f32a54ed48b84d70586f24cfd"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "b9a239583fb11d1ac83d6bb58ec476cc"
server_key_iv: "4453881f68bb17142feb01f1c8e55e16"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.639063 13721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "fc6ee47f32a54ed48b84d70586f24cfd"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "b9a239583fb11d1ac83d6bb58ec476cc"
server_key_iv: "4453881f68bb17142feb01f1c8e55e16"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.640395 13721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:06:49.641149 13736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:49.641456 13721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:49.641587 13721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "fc6ee47f32a54ed48b84d70586f24cfd"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "b9a239583fb11d1ac83d6bb58ec476cc"
server_key_iv: "4453881f68bb17142feb01f1c8e55e16"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.641671 13721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:49.653780 13721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:49.655902 13721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:49.656064 13721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:49.660002 13721 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41933
I20260923 15:06:49.660043 13788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41933 every 8 connection(s)
I20260923 15:06:49.660398 13721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:49.661288 13789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:49.663628 13789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd: Bootstrap starting.
I20260923 15:06:49.664220 13789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:49.664515 13789 log.cc:824] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:49.665254 13789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd: No bootstrap required, opened a new log
I20260923 15:06:49.667045 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13721
I20260923 15:06:49.667163 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:49.667359 8169 external_mini_cluster.cc:1428] Setting key 93881372159b3730e217419fa4ee5ce6
I20260923 15:06:49.667614 13789 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41933 } }
I20260923 15:06:49.667804 13789 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:49.667862 13789 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6ee47f32a54ed48b84d70586f24cfd, State: Initialized, Role: FOLLOWER
I20260923 15:06:49.667985 13789 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41933 } }
I20260923 15:06:49.668061 13789 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:49.668092 13789 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:49.668143 13789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:49.669126 13789 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41933 } }
I20260923 15:06:49.669255 13789 leader_election.cc:302] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc6ee47f32a54ed48b84d70586f24cfd; no voters:
I20260923 15:06:49.669405 13789 leader_election.cc:288] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:49.669476 13794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:49.669637 13789 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:49.669641 13794 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [term 1 LEADER]: Becoming Leader. State: Replica: fc6ee47f32a54ed48b84d70586f24cfd, State: Running, Role: LEADER
I20260923 15:06:49.669885 13794 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41933 } }
I20260923 15:06:49.670313 13796 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41933 } } }
I20260923 15:06:49.670487 13796 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:49.670739 13796 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc6ee47f32a54ed48b84d70586f24cfd. Latest consensus state: current_term: 1 leader_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6ee47f32a54ed48b84d70586f24cfd" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41933 } } }
I20260923 15:06:49.670974 13796 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:49.671583 13810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:49.671639 13810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:49.671756 13804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:49.672043 13804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:49.673985 13804 catalog_manager.cc:1347] Generated new cluster ID: f82050ca86fb4278908bf6c8ee4c30d5
I20260923 15:06:49.674053 13804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:49 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:49.680351 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:49.667911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.7.250.126:41933)
0923 15:06:49.668089 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:49.668098 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:49.668133 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:06:49.668258 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:49.668265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:49.668333 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:49.668478 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:49.669214 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:49.670762 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:49.671760 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:49.672345 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:49.672454 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:49.675725 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:49.675729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:49.675729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:49.675744 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:49.678494 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:49.678953 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:49.678957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:49.678957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:49.679081 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:49.679316 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:49.679317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:49.679317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:49.679650 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:49.679822 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:49.679943 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 15:06:49.680086 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260923 15:06:49.692607 13804 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:49.693313 13804 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:49.697569 13804 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc6ee47f32a54ed48b84d70586f24cfd: Generated new TSK 0
I20260923 15:06:49.697818 13804 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:06:49.725878 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41933
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:49.812331 13816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:49.812569 13816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:49.812604 13816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:49.813802 13816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:49.813871 13816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:49.813915 13816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:49.815537 13816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:49.815828 13816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:49.816066 13816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:49.821200 13825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:49.821314 13821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:49.821218 13822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:49.821445 13816 server_base.cc:1034] running on GCE node
I20260923 15:06:49.821720 13816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:49.821974 13816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:49.823112 13816 hybrid_clock.cc:648] HybridClock initialized: now 1790176009823096 us; error 38 us; skew 500 ppm
Sep 23 15:06:49 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:49.825798 13816 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:06:49.826598 13816 webserver.cc:460] Webserver started at http://127.7.250.65:36695/ using document root <none> and password file <none>
I20260923 15:06:49.826905 13816 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:49.826987 13816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:49.827169 13816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:49.828433 13816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "1d5ff009aad54b2f8927f64177434d32"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "9e0a58adcaab0355226382db41d655a6"
server_key_iv: "df1928cd4a47b01660710eb1cd9e0f65"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.828857 13816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "1d5ff009aad54b2f8927f64177434d32"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "9e0a58adcaab0355226382db41d655a6"
server_key_iv: "df1928cd4a47b01660710eb1cd9e0f65"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.830618 13816 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 15:06:49.831700 13832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:49.832039 13816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:49.832198 13816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "1d5ff009aad54b2f8927f64177434d32"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "9e0a58adcaab0355226382db41d655a6"
server_key_iv: "df1928cd4a47b01660710eb1cd9e0f65"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.832296 13816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:49.844805 13816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:49.847167 13816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:49.847342 13816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:49.847628 13816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:49.847942 13816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:49.847997 13816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:49.848039 13816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:49.848083 13816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:49.853900 13816 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43759
I20260923 15:06:49.853986 13946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43759 every 8 connection(s)
I20260923 15:06:49.854377 13816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:06:49 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:49.861585 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13816
I20260923 15:06:49.861685 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:49.861934 8169 external_mini_cluster.cc:1428] Setting key b4207287e081297f0849a8f16bfc7f8c
I20260923 15:06:49.864787 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:49.855223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:49753 (local address 127.7.250.126:41933)
0923 15:06:49.855308 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:49.855311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:49.855489 (+ 178us) server_negotiation.cc:407] Connection header received
0923 15:06:49.856500 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:49.856501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:49.856519 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:49.856572 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:49.857239 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:49.857758 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:49.858854 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:49.859077 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:49.859152 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:49.861278 (+ 2126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:49.861279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:49.861280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:49.861288 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:49.863347 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:49.863750 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:49.863752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:49.863752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:49.863817 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:49.864077 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:49.864078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:49.864079 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:49.864239 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:49.864308 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:49.864648 (+ 340us) server_negotiation.cc:299] Negotiation successful
0923 15:06:49.864723 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 15:06:49.865110 13949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:49.855148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41933 (local address 127.7.250.65:49753)
0923 15:06:49.855343 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:49.855395 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:49.856382 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:49.856642 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:49.856647 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:49.856938 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:49.857130 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:49.857134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:49.857856 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:49.857857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:49.858747 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:49.858750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:49.859172 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:49.859174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:49.859301 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:49.859782 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:49.859790 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:49.861173 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:49.863436 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:49.863443 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:49.863445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:49.863674 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:49.863881 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:49.863881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:49.863882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:49.863981 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:49.864311 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:49.864317 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:49.864494 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 15:06:49.864618 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 15:06:49.864807 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260923 15:06:49.865689 13947 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41933
I20260923 15:06:49.865793 13947 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:49.866001 13947 heartbeater.cc:511] Master 127.7.250.126:41933 requested a full tablet report, sending...
I20260923 15:06:49.866621 13753 ts_manager.cc:194] Registered new tserver with Master: 1d5ff009aad54b2f8927f64177434d32 (127.7.250.65:43759)
I20260923 15:06:49.867254 13753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:49753
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:06:49.890208 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41933
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:49.971649 13952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:49.971880 13952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:49.971918 13952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:49.973047 13952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:49.973110 13952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:49.973155 13952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:49.974751 13952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:49.975009 13952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:49.975237 13952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:49.980340 13957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:49.980340 13958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:49.980404 13960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:49.980535 13952 server_base.cc:1034] running on GCE node
I20260923 15:06:49.980836 13952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:49.981071 13952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:49.982240 13952 hybrid_clock.cc:648] HybridClock initialized: now 1790176009982202 us; error 52 us; skew 500 ppm
Sep 23 15:06:49 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:49.984709 13952 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:06:49.985217 13952 webserver.cc:460] Webserver started at http://127.7.250.66:33445/ using document root <none> and password file <none>
I20260923 15:06:49.985469 13952 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:49.985538 13952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:49.985688 13952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:49.986860 13952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "3ee9e48de84f4bd7b3af4feeb2042e08"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "6e03de3dcc1a94c2fbf0f294201b72ef"
server_key_iv: "0e0bde6cab90e9acc1724ccc5f136b51"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.987217 13952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "3ee9e48de84f4bd7b3af4feeb2042e08"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "6e03de3dcc1a94c2fbf0f294201b72ef"
server_key_iv: "0e0bde6cab90e9acc1724ccc5f136b51"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.988590 13952 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:06:49.989425 13967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:49.989643 13952 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:49.989748 13952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "3ee9e48de84f4bd7b3af4feeb2042e08"
format_stamp: "Formatted at 2026-09-23 15:06:49 on dist-test-slave-bb3x"
server_key: "6e03de3dcc1a94c2fbf0f294201b72ef"
server_key_iv: "0e0bde6cab90e9acc1724ccc5f136b51"
server_key_version: "encryptionkey@0"
I20260923 15:06:49.989850 13952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:50.007763 13952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:50.009985 13952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:50.010166 13952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:50.010470 13952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:50.010795 13952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:50.010857 13952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:50.010897 13952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:50.010926 13952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:50.016949 13952 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:34225
I20260923 15:06:50.016994 14081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:34225 every 8 connection(s)
I20260923 15:06:50.017338 13952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176009, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:50.025360 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 13952
I20260923 15:06:50.025487 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:50.025684 8169 external_mini_cluster.cc:1428] Setting key 4429f417e630bee8d1dad8be0a3158c5
I20260923 15:06:50.028296 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.018303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:53143 (local address 127.7.250.126:41933)
0923 15:06:50.018361 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.018364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.018620 (+ 256us) server_negotiation.cc:407] Connection header received
0923 15:06:50.019594 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.019596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.019609 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.019675 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.020285 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.020799 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.021786 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.021995 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.022097 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.024440 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:50.024442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:50.024442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:50.024450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:50.026425 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.027038 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.027040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.027040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.027125 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.027523 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.027524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.027525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.027654 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:50.027750 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:50.028112 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 15:06:50.028204 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:06:50.028506 14084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.018337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41933 (local address 127.7.250.66:53143)
0923 15:06:50.018499 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:50.018537 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:50.019479 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:50.019749 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:50.019752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:50.020003 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:50.020190 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:50.020194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.020895 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.020896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:50.021682 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:50.021684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.022101 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.022103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:50.022229 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.022824 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:50.022832 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:50.024294 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:50.026512 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:50.026535 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:50.026538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:50.026875 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:50.027270 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:50.027271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:50.027271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:50.027419 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:50.027731 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:50.027737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:50.027923 (+ 186us) client_negotiation.cc:769] Sending connection context
0923 15:06:50.028083 (+ 160us) client_negotiation.cc:240] Negotiation successful
0923 15:06:50.028251 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":90,"threads_started":1}
I20260923 15:06:50.028959 14082 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41933
I20260923 15:06:50.029047 14082 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:50.029244 14082 heartbeater.cc:511] Master 127.7.250.126:41933 requested a full tablet report, sending...
I20260923 15:06:50.029666 13753 ts_manager.cc:194] Registered new tserver with Master: 3ee9e48de84f4bd7b3af4feeb2042e08 (127.7.250.66:34225)
I20260923 15:06:50.030215 13753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:53143
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:06:50.053526 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41933
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:50.145112 14087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:50.145359 14087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:50.145385 14087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:50.146648 14087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:50.146693 14087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:50.146723 14087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:50.149255 14087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:50.149549 14087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:50.149847 14087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:50.156970 14092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:50.157301 14087 server_base.cc:1034] running on GCE node
W20260923 15:06:50.157006 14093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:50.157238 14095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:50.157601 14087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:50.157858 14087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:50.159003 14087 hybrid_clock.cc:648] HybridClock initialized: now 1790176010158991 us; error 28 us; skew 500 ppm
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.161562 14087 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:06:50.161993 14087 webserver.cc:460] Webserver started at http://127.7.250.67:43759/ using document root <none> and password file <none>
I20260923 15:06:50.162309 14087 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:50.162382 14087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:50.162540 14087 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:50.163594 14087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "6e279b16646644feb358093abe2e118f"
format_stamp: "Formatted at 2026-09-23 15:06:50 on dist-test-slave-bb3x"
server_key: "895ad4dca9e27cc39d57fbb78008f992"
server_key_iv: "9404f30cc0a3ee961168fbc9966c8114"
server_key_version: "encryptionkey@0"
I20260923 15:06:50.163965 14087 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "6e279b16646644feb358093abe2e118f"
format_stamp: "Formatted at 2026-09-23 15:06:50 on dist-test-slave-bb3x"
server_key: "895ad4dca9e27cc39d57fbb78008f992"
server_key_iv: "9404f30cc0a3ee961168fbc9966c8114"
server_key_version: "encryptionkey@0"
I20260923 15:06:50.165369 14087 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:50.166143 14102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:50.166390 14087 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:50.166467 14087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "6e279b16646644feb358093abe2e118f"
format_stamp: "Formatted at 2026-09-23 15:06:50 on dist-test-slave-bb3x"
server_key: "895ad4dca9e27cc39d57fbb78008f992"
server_key_iv: "9404f30cc0a3ee961168fbc9966c8114"
server_key_version: "encryptionkey@0"
I20260923 15:06:50.166549 14087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:50.176468 14087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:50.178829 14087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:50.178954 14087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:50.179141 14087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:50.179473 14087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:50.179531 14087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:50.179574 14087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:50.179600 14087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:50.185372 14087 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:45373
I20260923 15:06:50.185412 14216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:45373 every 8 connection(s)
I20260923 15:06:50.185768 14087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:50.188740 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14087
I20260923 15:06:50.188876 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:50.189117 8169 external_mini_cluster.cc:1428] Setting key a370fef683c856e9b77dd19daa22d3b8
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:06:50.197054 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.186568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:42705 (local address 127.7.250.126:41933)
0923 15:06:50.186614 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.186616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.186811 (+ 195us) server_negotiation.cc:407] Connection header received
0923 15:06:50.187798 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.187799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.187810 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.187881 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.188583 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.189648 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.191055 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.191260 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.191333 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.193984 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:50.193985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:50.193985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:50.193993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:50.195473 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.195895 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.195896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.195896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.195950 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.196414 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.196415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.196415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.196535 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:50.196606 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:50.196923 (+ 317us) server_negotiation.cc:299] Negotiation successful
0923 15:06:50.196989 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:50.197369 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.186529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41933 (local address 127.7.250.67:42705)
0923 15:06:50.186694 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:50.186737 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:50.187687 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:50.187955 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:50.187959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:50.188212 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:50.188447 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:50.188451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.189808 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.189809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:50.190976 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:50.190979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.191370 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.191372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:50.191490 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.192137 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:50.192147 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:50.193892 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:50.195565 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:50.195571 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:50.195573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:50.195816 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:50.196039 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:50.196040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:50.196040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:50.196230 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:50.196634 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:50.196641 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:50.196793 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 15:06:50.196894 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 15:06:50.197026 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260923 15:06:50.197921 14217 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41933
I20260923 15:06:50.198056 14217 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:50.198303 14217 heartbeater.cc:511] Master 127.7.250.126:41933 requested a full tablet report, sending...
I20260923 15:06:50.198767 13753 ts_manager.cc:194] Registered new tserver with Master: 6e279b16646644feb358093abe2e118f (127.7.250.67:45373)
I20260923 15:06:50.199141 13753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:42705
I20260923 15:06:50.201339 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/test-admin.keytab.
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.218830 8169 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 15:06:50.220687 14224 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:50.219854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38934 (local address 127.7.250.65:43759)
0923 15:06:50.220025 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.220030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.220040 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:50.220066 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.220067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.220081 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.220172 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.220580 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38934: BlockingRecv error: recv got EOF from 127.0.0.1:38934 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
W20260923 15:06:50.220856 8169 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc' of type FILE
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.222013 8169 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.223279 8169 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:50.232031 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.223816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38938 (local address 127.7.250.65:43759)
0923 15:06:50.223883 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.223885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.223928 (+ 43us) server_negotiation.cc:407] Connection header received
0923 15:06:50.224050 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.224051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.224063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.224126 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.224562 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.225374 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.226195 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.226660 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.226721 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.229121 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:50.229129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:50.229130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:50.229137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:50.230921 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.231307 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.231310 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.231310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.231363 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.231530 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.231531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.231531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.231665 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:50.231773 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:50.231893 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 15:06:50.231955 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260923 15:06:50.234040 14224 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:50.233422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38950 (local address 127.7.250.65:43759)
0923 15:06:50.233506 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.233508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.233517 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:50.233582 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.233583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.233593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.233650 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.233988 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38950: BlockingRecv error: recv got EOF from 127.0.0.1:38950 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260923 15:06:50.234076 8169 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc)
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.235213 8169 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.236081 8169 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:50.243445 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.236593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38962 (local address 127.7.250.65:43759)
0923 15:06:50.236660 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.236661 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.236670 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:50.236738 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.236738 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.236747 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.236802 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.237157 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.237697 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.238526 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.238726 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.238796 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.240983 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:50.240984 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:50.240984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:50.240992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:50.242397 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.242719 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.242720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.242721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.242796 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.242952 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.242952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.242952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.243053 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:50.243132 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:50.243210 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 15:06:50.243287 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260923 15:06:50.244990 14224 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:50.244475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38970 (local address 127.7.250.65:43759)
0923 15:06:50.244510 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.244512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.244629 (+ 117us) server_negotiation.cc:407] Connection header received
0923 15:06:50.244653 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.244654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.244662 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.244706 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.244946 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38970: BlockingRecv error: recv got EOF from 127.0.0.1:38970 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:06:50.245079 8169 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175983307587-8169-0/krb5kdc/krb5cc' of type FILE
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.246078 8169 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:50.246974 8169 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[13708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:06:50.254531 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:50.247461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38972 (local address 127.7.250.65:43759)
0923 15:06:50.247545 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:50.247547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:50.247556 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:50.247615 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:50.247616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:50.247624 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:50.247679 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:50.248198 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.248739 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.249440 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:50.249634 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:50.249687 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:50.251799 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:50.251800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:50.251800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:50.251806 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:50.253331 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.253638 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.253638 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.253639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.253695 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:50.253883 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:50.253884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:50.253884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:50.253983 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:50.254062 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:50.254144 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 15:06:50.254201 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 15:06:50.255365 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13816
I20260923 15:06:50.260753 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13952
I20260923 15:06:50.265240 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14087
I20260923 15:06:50.269893 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 13721
2026-09-23T15:06:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3478 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[14237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[14237](info): set up 2 sockets
Sep 23 15:06:50 dist-test-slave-bb3x krb5kdc[14237](info): commencing operation
krb5kdc: starting...
W20260923 15:06:51.312172 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab.
Entry for principal oryx/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab.
I20260923 15:06:51.433317 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:38959
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab
--principal=oryx/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:38959
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:51.512511 14250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:51.512760 14250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:51.512797 14250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:51.513963 14250 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:51.514035 14250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:51.514058 14250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:51.514079 14250 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:51.514109 14250 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:51.515650 14250 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:38959
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab
--principal=oryx/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:38959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:51.515928 14250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:51.516157 14250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:51.520965 14258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:51.520980 14256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:51.520974 14255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:51.521308 14250 server_base.cc:1034] running on GCE node
I20260923 15:06:51.521576 14250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:51.521873 14250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:51.523013 14250 hybrid_clock.cc:648] HybridClock initialized: now 1790176011522986 us; error 51 us; skew 500 ppm
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:51.530189 14250 init.cc:377] Logged in from keytab as oryx/127.7.250.126@KRBTEST.COM (short username oryx)
I20260923 15:06:51.530714 14250 webserver.cc:460] Webserver started at http://127.7.250.126:33917/ using document root <none> and password file <none>
I20260923 15:06:51.530915 14250 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:51.530956 14250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:51.531072 14250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:51.532145 14250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "af12aef1b44a4ed8beecad3a7b06b6b1"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "f7bed0b4f8a94ce3a82a8141806572b8"
server_key_iv: "b04ad11b5eb4337d9a373992bc6b7ed8"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.532444 14250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "af12aef1b44a4ed8beecad3a7b06b6b1"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "f7bed0b4f8a94ce3a82a8141806572b8"
server_key_iv: "b04ad11b5eb4337d9a373992bc6b7ed8"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.533735 14250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:51.534636 14265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.534884 14250 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:51.535015 14250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "af12aef1b44a4ed8beecad3a7b06b6b1"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "f7bed0b4f8a94ce3a82a8141806572b8"
server_key_iv: "b04ad11b5eb4337d9a373992bc6b7ed8"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.535097 14250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:51.549860 14250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:51.552093 14250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:51.552265 14250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:51.556465 14250 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:38959
I20260923 15:06:51.556516 14317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:38959 every 8 connection(s)
I20260923 15:06:51.556906 14250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:51.557899 14318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:51.558321 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14250
I20260923 15:06:51.558476 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:51.558678 8169 external_mini_cluster.cc:1428] Setting key dd94fa9ed28366c98200ab6baa4f5892
I20260923 15:06:51.560972 14318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Bootstrap starting.
I20260923 15:06:51.561659 14318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:51.561980 14318 log.cc:824] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:51.562822 14318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: No bootstrap required, opened a new log
I20260923 15:06:51.564924 14318 raft_consensus.cc:348] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } }
I20260923 15:06:51.565057 14318 raft_consensus.cc:374] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:51.565079 14318 raft_consensus.cc:729] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af12aef1b44a4ed8beecad3a7b06b6b1, State: Initialized, Role: FOLLOWER
I20260923 15:06:51.565218 14318 consensus_queue.cc:260] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } }
I20260923 15:06:51.565304 14318 raft_consensus.cc:388] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:51.565357 14318 raft_consensus.cc:482] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:51.565407 14318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 0 FOLLOWER]: Advancing to term 1
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:51.565981 14318 raft_consensus.cc:504] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } }
I20260923 15:06:51.566103 14318 leader_election.cc:302] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af12aef1b44a4ed8beecad3a7b06b6b1; no voters:
I20260923 15:06:51.566300 14318 leader_election.cc:288] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:51.566331 14323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:51.566478 14323 raft_consensus.cc:686] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 LEADER]: Becoming Leader. State: Replica: af12aef1b44a4ed8beecad3a7b06b6b1, State: Running, Role: LEADER
I20260923 15:06:51.566577 14318 sys_catalog.cc:564] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:51.566666 14323 consensus_queue.cc:237] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } }
I20260923 15:06:51.567006 14325 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } }
I20260923 15:06:51.567103 14325 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:51.566992 14323 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af12aef1b44a4ed8beecad3a7b06b6b1. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } }
I20260923 15:06:51.567433 14323 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:51.567625 14331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:51.568213 14331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:51.570011 14331 catalog_manager.cc:1347] Generated new cluster ID: c0ae394f1adb4c1b9dd72b947907434a
I20260923 15:06:51.570104 14331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:51.570452 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:51.559274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.7.250.126:38959)
0923 15:06:51.559465 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:51.559474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:51.559509 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:06:51.560028 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:51.560036 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:51.560095 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:51.560257 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:51.560998 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.562424 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.563217 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.563822 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.563948 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:51.566530 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:51.566535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:51.566535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:51.566550 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:51.568680 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:51.569082 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:51.569085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:51.569085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:51.569188 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:51.569436 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:51.569437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:51.569438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:51.569720 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:51.569895 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:51.570020 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 15:06:51.570161 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":82,"threads_started":1}
I20260923 15:06:51.579615 14331 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:51.580327 14331 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:51.584307 14331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Generated new TSK 0
I20260923 15:06:51.584481 14331 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab.
Entry for principal oryx/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab.
I20260923 15:06:51.617058 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:38959
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:51.707460 14344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:51.707666 14344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:51.707691 14344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:51.708856 14344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:51.708900 14344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:51.708927 14344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:51.710470 14344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:38959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:51.710767 14344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:51.711092 14344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:51.717187 14352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:51.717178 14350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:51.717168 14349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:51.717517 14344 server_base.cc:1034] running on GCE node
I20260923 15:06:51.717825 14344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:51.718101 14344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:51.719251 14344 hybrid_clock.cc:648] HybridClock initialized: now 1790176011719234 us; error 33 us; skew 500 ppm
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:51.721727 14344 init.cc:377] Logged in from keytab as oryx/127.7.250.65@KRBTEST.COM (short username oryx)
I20260923 15:06:51.722203 14344 webserver.cc:460] Webserver started at http://127.7.250.65:41693/ using document root <none> and password file <none>
I20260923 15:06:51.722497 14344 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:51.722546 14344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:51.722707 14344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:51.723869 14344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "c412d6a53bf84c72a8433a17e118bd55"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "6eece13df4cd2cf3b4083adbe66c8616"
server_key_iv: "02720f9b8177f5b41a50c94161c2924e"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.724232 14344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "c412d6a53bf84c72a8433a17e118bd55"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "6eece13df4cd2cf3b4083adbe66c8616"
server_key_iv: "02720f9b8177f5b41a50c94161c2924e"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.725659 14344 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:51.726555 14359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.726724 14344 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:51.726828 14344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "c412d6a53bf84c72a8433a17e118bd55"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "6eece13df4cd2cf3b4083adbe66c8616"
server_key_iv: "02720f9b8177f5b41a50c94161c2924e"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.726910 14344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:51.747465 14344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:51.749652 14344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:51.749825 14344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:51.750083 14344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:51.750469 14344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:51.750524 14344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.750560 14344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:51.750587 14344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.756618 14344 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43863
I20260923 15:06:51.756662 14473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43863 every 8 connection(s)
I20260923 15:06:51.756971 14344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:51.762789 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14344
I20260923 15:06:51.762910 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:51.763141 8169 external_mini_cluster.cc:1428] Setting key 44c6cb17dee706d99e2210f1cc46ac3c
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:51.766691 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:51.757635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:59367 (local address 127.7.250.126:38959)
0923 15:06:51.757697 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:51.757699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:51.757920 (+ 221us) server_negotiation.cc:407] Connection header received
0923 15:06:51.758498 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:51.758499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:51.758511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:51.758571 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:51.759525 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.760052 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.761022 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.761222 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.761274 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:51.763722 (+ 2448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:51.763723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:51.763724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:51.763731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:51.765187 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:51.765602 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:51.765603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:51.765604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:51.765663 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:51.765943 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:51.765944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:51.765945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:51.766064 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:51.766141 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:51.766509 (+ 368us) server_negotiation.cc:299] Negotiation successful
0923 15:06:51.766600 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:06:51.766889 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:51.757599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:59367)
0923 15:06:51.757786 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:51.757827 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:51.758401 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:51.758643 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:51.758647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:51.759228 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:51.759423 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:51.759428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.760141 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.760141 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:51.760945 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:51.760947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.761298 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.761299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:51.761394 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:51.761933 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:51.761941 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:51.763632 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:51.765274 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:51.765282 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:51.765284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:51.765519 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:51.765745 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:51.765746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:51.765746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:51.765862 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:51.766132 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:51.766139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:51.766357 (+ 218us) client_negotiation.cc:769] Sending connection context
0923 15:06:51.766484 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:06:51.766643 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":110,"threads_started":1}
I20260923 15:06:51.767361 14474 heartbeater.cc:347] Connected to a master server at 127.7.250.126:38959
I20260923 15:06:51.767462 14474 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:51.767639 14474 heartbeater.cc:511] Master 127.7.250.126:38959 requested a full tablet report, sending...
I20260923 15:06:51.768167 14282 ts_manager.cc:194] Registered new tserver with Master: c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863)
I20260923 15:06:51.768818 14282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:59367
WARNING: no policy specified for oryx/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab.
Entry for principal oryx/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab.
I20260923 15:06:51.790674 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:38959
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:51.873068 14479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:51.873271 14479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:51.873296 14479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:51.874462 14479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:51.874503 14479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:51.874531 14479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:51.875952 14479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:38959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:51.876140 14479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:51.876322 14479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:51.881551 14485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:51.881644 14479 server_base.cc:1034] running on GCE node
W20260923 15:06:51.881547 14487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:51.881551 14484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:51.882048 14479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:51.882356 14479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:51.883533 14479 hybrid_clock.cc:648] HybridClock initialized: now 1790176011883526 us; error 33 us; skew 500 ppm
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:51.886142 14479 init.cc:377] Logged in from keytab as oryx/127.7.250.66@KRBTEST.COM (short username oryx)
I20260923 15:06:51.886663 14479 webserver.cc:460] Webserver started at http://127.7.250.66:34663/ using document root <none> and password file <none>
I20260923 15:06:51.886912 14479 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:51.886958 14479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:51.887082 14479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:51.888118 14479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "e72bdea6a7684fde9af7a8c65f28ed01"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "55c05b6fbf5ad2623600e030148a58b7"
server_key_iv: "6b2f629e7fd4f0090319d0bd115c88d3"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.888482 14479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "e72bdea6a7684fde9af7a8c65f28ed01"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "55c05b6fbf5ad2623600e030148a58b7"
server_key_iv: "6b2f629e7fd4f0090319d0bd115c88d3"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.889875 14479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:51.890772 14495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.890978 14479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:51.891076 14479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "e72bdea6a7684fde9af7a8c65f28ed01"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "55c05b6fbf5ad2623600e030148a58b7"
server_key_iv: "6b2f629e7fd4f0090319d0bd115c88d3"
server_key_version: "encryptionkey@0"
I20260923 15:06:51.891144 14479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:51.907289 14479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:51.909579 14479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:51.909696 14479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:51.909917 14479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:51.910192 14479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:51.910223 14479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.910247 14479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:51.910331 14479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:51.916090 14479 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:35141
I20260923 15:06:51.916185 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:35141 every 8 connection(s)
I20260923 15:06:51.916540 14479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:51 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:51.925925 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14479
I20260923 15:06:51.926064 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:51.926388 8169 external_mini_cluster.cc:1428] Setting key 7fea71459570f8481c2aca1a3ea0729d
I20260923 15:06:51.926752 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:51.917436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:43853 (local address 127.7.250.126:38959)
0923 15:06:51.917513 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:51.917515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:51.917733 (+ 218us) server_negotiation.cc:407] Connection header received
0923 15:06:51.918204 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:51.918205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:51.918216 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:51.918352 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:51.919255 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.919793 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.920768 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.920960 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.921018 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:51.923315 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:51.923317 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:51.923317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:51.923326 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:51.924938 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:51.925396 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:51.925398 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:51.925398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:51.925463 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:51.925782 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:51.925783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:51.925783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:51.925952 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:51.926080 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:51.926550 (+ 470us) server_negotiation.cc:299] Negotiation successful
0923 15:06:51.926652 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:06:51.927045 14611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:51.917407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:43853)
0923 15:06:51.917576 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:51.917619 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:51.918119 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:51.918415 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:51.918419 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:51.918980 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:51.919172 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:51.919177 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.919875 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.919876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:51.920687 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:51.920689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:51.921058 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:51.921059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:51.921168 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:51.921693 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:51.921701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:51.923215 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:51.925038 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:51.925045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:51.925047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:51.925302 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:51.925547 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:51.925548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:51.925548 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:51.925654 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:51.926080 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:51.926088 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:51.926380 (+ 292us) client_negotiation.cc:769] Sending connection context
0923 15:06:51.926507 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:06:51.926700 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":100,"threads_started":1}
I20260923 15:06:51.927592 14609 heartbeater.cc:347] Connected to a master server at 127.7.250.126:38959
I20260923 15:06:51.927716 14609 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:51.927932 14609 heartbeater.cc:511] Master 127.7.250.126:38959 requested a full tablet report, sending...
I20260923 15:06:51.928496 14282 ts_manager.cc:194] Registered new tserver with Master: e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:51.928942 14282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:43853
WARNING: no policy specified for oryx/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab.
Entry for principal oryx/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab.
I20260923 15:06:51.958168 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:38959
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:52.049412 14614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:52.049626 14614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:52.049649 14614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:52.050853 14614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:52.050895 14614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:52.050922 14614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:52.052286 14614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:38959
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:52.052469 14614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:52.052639 14614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:52.057693 14622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.057713 14619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.057736 14620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.057991 14614 server_base.cc:1034] running on GCE node
I20260923 15:06:52.058203 14614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.058516 14614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.059656 14614 hybrid_clock.cc:648] HybridClock initialized: now 1790176012059640 us; error 34 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.062451 14614 init.cc:377] Logged in from keytab as oryx/127.7.250.67@KRBTEST.COM (short username oryx)
I20260923 15:06:52.062985 14614 webserver.cc:460] Webserver started at http://127.7.250.67:46481/ using document root <none> and password file <none>
I20260923 15:06:52.063246 14614 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.063313 14614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.063459 14614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:52.064513 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "cf6d0e8ef4a746599484cb9dc4f217cd"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "697001d5c6337be57babb8b51844e135"
server_key_iv: "6a82a96227b5183e6627410fa4720629"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.064863 14614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "cf6d0e8ef4a746599484cb9dc4f217cd"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "697001d5c6337be57babb8b51844e135"
server_key_iv: "6a82a96227b5183e6627410fa4720629"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.066388 14614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260923 15:06:52.067408 14630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.067593 14614 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:52.067716 14614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "cf6d0e8ef4a746599484cb9dc4f217cd"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "697001d5c6337be57babb8b51844e135"
server_key_iv: "6a82a96227b5183e6627410fa4720629"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.067812 14614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.078404 14614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.080281 14614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:52.080461 14614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.080698 14614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:52.081002 14614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:52.081058 14614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.081101 14614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:52.081125 14614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.087299 14614 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:42125
I20260923 15:06:52.087340 14743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:42125 every 8 connection(s)
I20260923 15:06:52.087656 14614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:52.093544 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14614
I20260923 15:06:52.093670 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:52.093984 8169 external_mini_cluster.cc:1428] Setting key 435a2bffec1951cf5181929f326ecb1f
I20260923 15:06:52.097426 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.088390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:46173 (local address 127.7.250.126:38959)
0923 15:06:52.088460 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.088462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.088657 (+ 195us) server_negotiation.cc:407] Connection header received
0923 15:06:52.089168 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.089169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.089180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.089231 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.090402 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.090898 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.091863 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.092045 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.092098 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.094637 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.094639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.094639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.094648 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.096168 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.096574 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.096575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.096575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.096625 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.096816 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.096817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.096817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.096919 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.096971 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.097290 (+ 319us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.097362 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:06:52.097662 14746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.088392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:46173)
0923 15:06:52.088557 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.088591 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.089081 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.089295 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.089299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.090011 (+ 712us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.090210 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.090215 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.090986 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.090987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.091786 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.091789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.092119 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.092120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.092220 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.092719 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.092728 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.094536 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.096261 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.096269 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.096271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.096504 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.096666 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.096666 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.096667 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.096763 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.097012 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.097019 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.097150 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.097267 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.097403 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":87,"threads_started":1}
I20260923 15:06:52.098335 14744 heartbeater.cc:347] Connected to a master server at 127.7.250.126:38959
I20260923 15:06:52.098438 14744 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.098592 14744 heartbeater.cc:511] Master 127.7.250.126:38959 requested a full tablet report, sending...
I20260923 15:06:52.099010 14282 ts_manager.cc:194] Registered new tserver with Master: cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125)
I20260923 15:06:52.099390 14282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:46173
I20260923 15:06:52.106045 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.250.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.125.keytab.
Entry for principal oryx/127.7.250.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.125.keytab.
I20260923 15:06:52.134322 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:33905
--webserver_interface=127.7.250.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.125.keytab
--principal=oryx/127.7.250.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:38959,127.7.250.125:33905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:52.218384 14749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:52.218608 14749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:52.218647 14749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:52.219820 14749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:52.219890 14749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:52.219918 14749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:52.219950 14749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:52.219973 14749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:52.221431 14749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:38959,127.7.250.125:33905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.125.keytab
--principal=oryx/127.7.250.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:33905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.250.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
W20260923 15:06:52.221570 14749 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.250.126:38959,127.7.250.125:33905'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 15:06:52.221724 14749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:52.221940 14749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:52.226809 14755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.226819 14754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.226817 14749 server_base.cc:1034] running on GCE node
W20260923 15:06:52.226903 14757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.227198 14749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.227463 14749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.228595 14749 hybrid_clock.cc:648] HybridClock initialized: now 1790176012228574 us; error 38 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.231212 14749 init.cc:377] Logged in from keytab as oryx/127.7.250.125@KRBTEST.COM (short username oryx)
I20260923 15:06:52.231663 14749 webserver.cc:460] Webserver started at http://127.7.250.125:34355/ using document root <none> and password file <none>
I20260923 15:06:52.231870 14749 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.231912 14749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.231999 14749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:52.232985 14749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/instance:
uuid: "f0834107645142c39194500863ee9642"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "d7bacdfc75227b6cbbad1bc4c13167d7"
server_key_iv: "6ab5fdbf5a8cef410970809347469229"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.233280 14749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal/instance:
uuid: "f0834107645142c39194500863ee9642"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "d7bacdfc75227b6cbbad1bc4c13167d7"
server_key_iv: "6ab5fdbf5a8cef410970809347469229"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.234751 14749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:52.235497 14765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.235731 14749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:52.235841 14749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
uuid: "f0834107645142c39194500863ee9642"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "d7bacdfc75227b6cbbad1bc4c13167d7"
server_key_iv: "6ab5fdbf5a8cef410970809347469229"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.235934 14749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.255489 14749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.257882 14749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:52.258055 14749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.262319 14749 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.125:33905
I20260923 15:06:52.262449 14817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.125:33905 every 8 connection(s)
I20260923 15:06:52.262769 14749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
I20260923 15:06:52.263767 14818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:52.265297 14818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:52.270108 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14749
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.125@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:52.275925 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.266139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.7.250.126:38959)
0923 15:06:52.266202 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.266204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.266440 (+ 236us) server_negotiation.cc:407] Connection header received
0923 15:06:52.266977 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.266979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.266992 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.267133 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.267939 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.268503 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.269512 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.269722 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.269796 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.272499 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.272500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.272500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.272509 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.274074 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.274758 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.274759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.274760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.274864 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.275111 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.275112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.275113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.275332 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.275425 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.275704 (+ 279us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.275793 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:06:52.275957 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.266082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:34072)
0923 15:06:52.266307 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.266348 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.266888 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.267221 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.267230 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.267539 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.267835 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.267840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.268598 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.268598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.269414 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.269416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.269817 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.269818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.269957 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.270569 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.270596 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.272411 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.274179 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.274185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.274186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.274643 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.274925 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.274925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.274926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.275028 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.275429 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.275434 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.275554 (+ 120us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.275645 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.275770 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":125,"threads_started":1}
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
I20260923 15:06:52.276657 14818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:52.280169 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.270894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.7.250.125:33905)
0923 15:06:52.271039 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.271044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.271063 (+ 19us) server_negotiation.cc:407] Connection header received
0923 15:06:52.271094 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.271096 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.271117 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.271201 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.272037 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.272964 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.273833 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.274369 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.274444 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.277216 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.277223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.277223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.277229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.278889 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.279283 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.279286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.279287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.279358 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.279559 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.279560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.279560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.279745 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.279942 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.279986 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.280059 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.125@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
I20260923 15:06:52.284005 14820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.276927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:52294)
0923 15:06:52.277000 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.277011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.277050 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.277317 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.277317 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.277460 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.277501 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.277503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.278141 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.278143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.278689 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.278691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.279069 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.279069 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.279132 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.279554 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.279559 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.280848 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.282994 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.282995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.282995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.283155 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.283375 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.283376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.283376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.283427 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.283665 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.283665 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.283702 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.283770 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.283866 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260923 15:06:52.284195 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.276993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52294 (local address 127.7.250.125:33905)
0923 15:06:52.277142 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.277144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.277150 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:06:52.277173 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.277173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.277180 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.277312 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.277581 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.278065 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.278771 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.278986 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.279069 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.280952 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.280953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.280954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.280959 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.282894 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.283241 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.283242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.283242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.283294 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.283492 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.283493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.283493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.283600 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.283674 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.283806 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.283948 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260923 15:06:52.285465 14818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Bootstrap starting.
I20260923 15:06:52.286065 14818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:52.286398 14818 log.cc:824] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:52.287043 14818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: No bootstrap required, opened a new log
I20260923 15:06:52.288426 14818 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } }
I20260923 15:06:52.288547 14818 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:52.288569 14818 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0834107645142c39194500863ee9642, State: Initialized, Role: FOLLOWER
I20260923 15:06:52.288687 14818 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } }
I20260923 15:06:52.288920 14825 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } } }
I20260923 15:06:52.288996 14818 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:52.289011 14825 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:06:52.290767 14836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:52.290829 14836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:52.292294 14749 master_runner.cc:428] Detected that this master f0834107645142c39194500863ee9642 is joining an existing cluster
I20260923 15:06:52.292344 14749 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.250.125:33905
I20260923 15:06:52.300627 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.293464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.7.250.126:38959)
0923 15:06:52.293517 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.293520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.293770 (+ 250us) server_negotiation.cc:407] Connection header received
0923 15:06:52.293800 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.293801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.293815 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.293884 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.294353 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.294888 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.295910 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.296225 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.296302 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.297451 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.297453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.297453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.297460 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.299173 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.299715 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.299716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.299717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.299773 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.300000 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.300001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.300001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.300141 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.300187 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.300409 (+ 222us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.300519 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:06:52.300992 14843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.293705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:52308)
0923 15:06:52.293837 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.293847 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.293913 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.294030 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.294031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.294219 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.294366 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.294368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.295287 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.295288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.296027 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.296028 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.296458 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.296459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.296531 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.297041 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.297046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.297499 (+ 453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.299627 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.299627 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.299628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.299905 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.300110 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.300111 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.300111 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.300166 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.300408 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.300409 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.300446 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.300514 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.300617 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
I20260923 15:06:52.301077 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.293394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:34076)
0923 15:06:52.293582 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.293594 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.293659 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.293897 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.293898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.294141 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.294219 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.294221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.294979 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.294980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.295822 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.295824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.296320 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.296321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.296391 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.296883 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.296888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.297345 (+ 457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.299261 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.299262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.299262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.299585 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.299844 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.299845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.299845 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.299912 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.300236 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.300237 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.300284 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.300716 (+ 432us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.300825 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":96,"threads_started":1}
I20260923 15:06:52.301126 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.293737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52308 (local address 127.7.250.125:33905)
0923 15:06:52.293774 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.293776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.293891 (+ 115us) server_negotiation.cc:407] Connection header received
0923 15:06:52.293969 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.293970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.293977 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.294015 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.294448 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.295201 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.296118 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.296371 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.296447 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.297580 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.297581 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.297582 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.297588 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.299525 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.299990 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.299991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.299992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.300040 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.300235 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.300236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.300237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.300343 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.300416 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.300540 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.300839 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":22}
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.126@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
I20260923 15:06:52.311815 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.303956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52314 (local address 127.7.250.125:33905)
0923 15:06:52.303997 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.303999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.304123 (+ 124us) server_negotiation.cc:407] Connection header received
0923 15:06:52.304319 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.304321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.304331 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.304388 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.304896 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.305348 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.306128 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.306401 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.306494 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.308672 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.308694 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.308695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.308702 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.310379 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.310835 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.310836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.310837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.310897 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.311122 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.311123 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.311123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.311317 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.311407 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.311620 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.311707 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:06:52.311831 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.303873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:52314)
0923 15:06:52.304008 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.304052 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.304149 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.304396 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.304399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.304750 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.304802 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.304804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.305430 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.305431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.306037 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.306039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.306497 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.306498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.306585 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.307057 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.307062 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.308581 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.310488 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.310497 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.310499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.310734 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.310979 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.310980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.310980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.311050 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.311399 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.311407 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.311465 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.311574 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.311715 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":73,"threads_started":1}
I20260923 15:06:52.312428 14282 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.250.125:33905: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f0834107645142c39194500863ee9642"
member_type: NON_VOTER
last_known_addr {
host: "127.7.250.125"
port: 33905
}
attrs {
promote: true
}
}
dest_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1"
cas_config_opid_index: -1
I20260923 15:06:52.312855 14282 consensus_queue.cc:237] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } }
I20260923 15:06:52.313336 14846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f0834107645142c39194500863ee9642 (127.7.250.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.313733 14792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:52.314662 14792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Refusing update from remote peer af12aef1b44a4ed8beecad3a7b06b6b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 15:06:52.315047 14845 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:52.315780 14846 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.250.125:33905
I20260923 15:06:52.316025 14845 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.316161 14749 master.cc:537] Master@127.7.250.125:33905 shutting down...
I20260923 15:06:52.316192 14845 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:52.317399 14848 mvcc.cc:205] Tried to move back new op lower bound from 7332560943387160576 to 7332560943377432576. Current Snapshot: MvccSnapshot[applied={T|T < 7332560943387160576}]
I20260923 15:06:52.317618 14792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f0834107645142c39194500863ee9642 (127.7.250.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.318761 14850 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.318943 14850 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:06:52.319002 14825 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af12aef1b44a4ed8beecad3a7b06b6b1. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.319031 14850 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.319072 14825 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:06:52.319087 14850 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:06:52.320544 14749 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Raft consensus shutting down.
I20260923 15:06:52.320677 14749 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Raft consensus is shut down!
I20260923 15:06:52.320744 14749 tablet_replica.cc:330] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: stopping tablet replica
I20260923 15:06:52.323909 14749 master.cc:552] Master@127.7.250.125:33905 shutdown complete.
I20260923 15:06:52.323959 14749 master_runner.cc:305] Clearing existing system tablet
I20260923 15:06:52.324088 14749 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.324136 14749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.324855 14749 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:52.325436 14856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.325574 14749 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.325637 14749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.325662 14749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
uuid: "f0834107645142c39194500863ee9642"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "d7bacdfc75227b6cbbad1bc4c13167d7"
server_key_iv: "6ab5fdbf5a8cef410970809347469229"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.326436 14749 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:52.327982 14749 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 15:06:52.328060 14749 log.cc:1196] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 15:06:52.328295 14749 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Deleting consensus metadata
I20260923 15:06:52.328419 14749 master_runner.cc:315] Copying system tablet from 127.7.250.126:38959
I20260923 15:06:52.329061 14749 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: tablet copy: Beginning tablet copy session from remote peer at address 127.7.250.126:38959
I20260923 15:06:52.335892 14862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.329430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:34090)
0923 15:06:52.329575 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.329589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.329657 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.329840 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.329841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.329994 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.330050 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.330053 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.331035 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.331037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.331754 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.331757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.332239 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.332240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.332302 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.332812 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.332817 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.333126 (+ 309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.334980 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.334981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.334982 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.335144 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.335338 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.335339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.335339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.335389 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.335642 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.335642 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.335690 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.335746 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.335823 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":102,"threads_started":1}
I20260923 15:06:52.335968 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.329514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34090 (local address 127.7.250.126:38959)
0923 15:06:52.329558 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.329560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.329656 (+ 96us) server_negotiation.cc:407] Connection header received
0923 15:06:52.329765 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.329766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.329778 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.329834 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.330122 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.330928 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.331850 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.332162 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.332228 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.333217 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.333219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.333219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.333227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.334886 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.335212 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.335214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.335214 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.335274 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.335449 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.335450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.335450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.335579 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.335647 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.335792 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.335871 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:52.336287 14302 tablet_copy_service.cc:140] P af12aef1b44a4ed8beecad3a7b06b6b1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f0834107645142c39194500863ee9642 ({username='oryx', principal='oryx/127.7.250.125@KRBTEST.COM'} at 127.0.0.1:34090)
I20260923 15:06:52.336413 14302 tablet_copy_service.cc:161] P af12aef1b44a4ed8beecad3a7b06b6b1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f0834107645142c39194500863ee9642 at {username='oryx', principal='oryx/127.7.250.125@KRBTEST.COM'} at 127.0.0.1:34090: session id = f0834107645142c39194500863ee9642-00000000000000000000000000000000
I20260923 15:06:52.337302 14302 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 15:06:52.338001 14749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 15:06:52.339428 14749 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: tablet copy: Starting download of 0 data blocks...
I20260923 15:06:52.339581 14749 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: tablet copy: Starting download of 1 WAL segments...
I20260923 15:06:52.340481 14749 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 15:06:52.341476 14302 tablet_copy_service.cc:342] P af12aef1b44a4ed8beecad3a7b06b6b1: Request end of tablet copy session f0834107645142c39194500863ee9642-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.250.125@KRBTEST.COM'} at 127.0.0.1:34090
I20260923 15:06:52.341539 14302 tablet_copy_service.cc:433] P af12aef1b44a4ed8beecad3a7b06b6b1: ending tablet copy session f0834107645142c39194500863ee9642-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f0834107645142c39194500863ee9642
I20260923 15:06:52.344238 14749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:52.345549 14863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.345595 14866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.345721 14864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.345978 14749 server_base.cc:1034] running on GCE node
I20260923 15:06:52.346084 14749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.346249 14749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.347442 14749 hybrid_clock.cc:648] HybridClock initialized: now 1790176012347434 us; error 24 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.348934 14749 init.cc:377] Logged in from keytab as oryx/127.7.250.125@KRBTEST.COM (short username oryx)
I20260923 15:06:52.349270 14749 webserver.cc:460] Webserver started at http://127.7.250.125:36725/ using document root <none> and password file <none>
I20260923 15:06:52.349417 14749 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.349476 14749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.350086 14749 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:52.350711 14873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.350867 14749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.350926 14749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
uuid: "f0834107645142c39194500863ee9642"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "d7bacdfc75227b6cbbad1bc4c13167d7"
server_key_iv: "6ab5fdbf5a8cef410970809347469229"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.350986 14749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.359829 14749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.361430 14749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.364415 14749 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.125:33905
I20260923 15:06:52.364528 14925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.125:33905 every 8 connection(s)
I20260923 15:06:52.364782 14749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
I20260923 15:06:52.365520 14926 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:06:52.365936 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Bootstrap starting.
I20260923 15:06:52.368000 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:06:52.368204 14926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Bootstrap complete.
I20260923 15:06:52.368391 14926 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } }
I20260923 15:06:52.368479 14926 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f0834107645142c39194500863ee9642, State: Initialized, Role: LEARNER
I20260923 15:06:52.368531 14926 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } }
I20260923 15:06:52.368724 14926 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:52.368716 14929 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } }
I20260923 15:06:52.368824 14929 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:06:52.370090 14940 catalog_manager.cc:1259] Loaded cluster ID: c0ae394f1adb4c1b9dd72b947907434a
I20260923 15:06:52.370134 14940 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: loading cluster ID for follower catalog manager: success
I20260923 15:06:52.371359 14940 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: acquiring CA information for follower catalog manager: success
I20260923 15:06:52.371639 14940 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:06:52.389508 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.381862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52332 (local address 127.7.250.125:33905)
0923 15:06:52.381998 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.382000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.382008 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:52.382031 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.382032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.382041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.382114 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.382915 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.383590 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.384771 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.384956 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.385009 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.386615 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.386616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.386617 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.386623 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.388215 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.388658 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.388659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.388659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.388710 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.388937 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.388938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.388938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.389034 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.389106 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.389335 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.389400 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for oryx/127.7.250.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.124.keytab.
Entry for principal oryx/127.7.250.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.124.keytab.
I20260923 15:06:52.418838 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:40281
--webserver_interface=127.7.250.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.124.keytab
--principal=oryx/127.7.250.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:52.502393 14944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:52.502604 14944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:52.502626 14944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:52.503875 14944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:52.503921 14944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:52.503937 14944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:52.503948 14944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:52.503960 14944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:52.505558 14944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.124.keytab
--principal=oryx/127.7.250.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:40281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.250.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:52.505808 14944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:52.506075 14944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:52.511258 14944 server_base.cc:1034] running on GCE node
W20260923 15:06:52.511171 14950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.511166 14949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.511211 14952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.511726 14944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.511991 14944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.513149 14944 hybrid_clock.cc:648] HybridClock initialized: now 1790176012513114 us; error 76 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.515867 14944 init.cc:377] Logged in from keytab as oryx/127.7.250.124@KRBTEST.COM (short username oryx)
I20260923 15:06:52.516263 14944 webserver.cc:460] Webserver started at http://127.7.250.124:34457/ using document root <none> and password file <none>
I20260923 15:06:52.516505 14944 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.516573 14944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.516736 14944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:52.517825 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/instance:
uuid: "b23945c5404141028d4bca48daaba76b"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "f37ba330279185e301522429ce334423"
server_key_iv: "e1434a75b1a84956f4f523f6b1596cd8"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.518199 14944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal/instance:
uuid: "b23945c5404141028d4bca48daaba76b"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "f37ba330279185e301522429ce334423"
server_key_iv: "e1434a75b1a84956f4f523f6b1596cd8"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.519572 14944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:52.520413 14959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.520733 14944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:52.520852 14944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
uuid: "b23945c5404141028d4bca48daaba76b"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "f37ba330279185e301522429ce334423"
server_key_iv: "e1434a75b1a84956f4f523f6b1596cd8"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.520928 14944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.540169 14944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.542549 14944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:52.542725 14944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.546690 14944 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.124:40281
I20260923 15:06:52.546726 15011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.124:40281 every 8 connection(s)
I20260923 15:06:52.547093 14944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
I20260923 15:06:52.548028 15012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:52.549907 15012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:52.554091 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 14944
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.124@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.250.124@KRBTEST.COM
I20260923 15:06:52.560173 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.550845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34104 (local address 127.7.250.126:38959)
0923 15:06:52.550929 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.550931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.551019 (+ 88us) server_negotiation.cc:407] Connection header received
0923 15:06:52.551590 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.551592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.551603 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.551670 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.552361 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.552890 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.553994 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.554332 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.554412 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.556756 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.556757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.556758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.556766 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.558561 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.559032 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.559034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.559034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.559095 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.559448 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.559448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.559449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.559554 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.559608 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.559978 (+ 370us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.560086 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 15:06:52.560348 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.550666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:34104)
0923 15:06:52.550877 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.550926 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.551496 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.551727 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.551730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.552015 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.552224 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.552228 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.552974 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.552975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.553898 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.553901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.554451 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.554453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.554572 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.555157 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.555166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.556660 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.558652 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.558659 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.558661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.558952 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.559166 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.559167 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.559167 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.559366 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.559625 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.559633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.559803 (+ 170us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.559921 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.560078 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":103,"threads_started":1}
I20260923 15:06:52.561250 15012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:52.563859 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.554674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48238 (local address 127.7.250.124:40281)
0923 15:06:52.554858 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.554862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.554879 (+ 17us) server_negotiation.cc:407] Connection header received
0923 15:06:52.554907 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.554909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.554930 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.555018 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.555616 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.556306 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.557236 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.557702 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.558040 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.560501 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.560506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.560506 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.560513 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.562480 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.562861 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.562864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.562864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.562971 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.563227 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.563228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.563228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.563453 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.563596 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.563678 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.563755 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":126,"threads_started":1}
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.124@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
I20260923 15:06:52.569026 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.561580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:52348)
0923 15:06:52.561693 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.561705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.561759 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.561896 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.561897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.562050 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.562103 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.562105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.563250 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.563251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.564016 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.564018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.564553 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.564554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.564628 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.564955 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.564959 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.566126 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.567963 (+ 1837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.567964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.567964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.568128 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.568348 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.568349 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.568349 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.568406 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.568681 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.568682 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.568730 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.568817 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.568910 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260923 15:06:52.569097 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.561642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52348 (local address 127.7.250.125:33905)
0923 15:06:52.561712 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.561713 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.561745 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:06:52.561823 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.561824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.561835 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.561881 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.562315 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.562973 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.564088 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.564477 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.564553 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.566247 (+ 1694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.566248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.566249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.566277 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.567870 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.568223 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.568224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.568225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.568282 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.568474 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.568475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.568475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.568631 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.568689 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.568842 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.568981 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:06:52.569480 15012 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.124@KRBTEST.COM for oryx/127.7.250.124@KRBTEST.COM
I20260923 15:06:52.576112 15014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.569716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:48248)
0923 15:06:52.569750 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.569759 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.569795 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.569905 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.569906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.570007 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.570034 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.570035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.570904 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.570906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.571462 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.571464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.571861 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.571862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.571906 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.572266 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.572270 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.573558 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.575268 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.575269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.575270 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.575433 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.575618 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.575619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.575620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.575676 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.575877 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.575878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.575910 (+ 32us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.575960 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.576034 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 15:06:52.576259 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.569773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.7.250.124:40281)
0923 15:06:52.569805 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.569806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.569815 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:52.569857 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.569857 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.569865 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.569903 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.570126 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.570809 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.571545 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.571804 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.571857 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.573633 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.573635 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.573635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.573640 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.575180 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.575499 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.575500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.575500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.575545 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.575734 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.575735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.575735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.575823 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.575870 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.575974 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.576076 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":15}
I20260923 15:06:52.577559 15012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Bootstrap starting.
I20260923 15:06:52.578459 15012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:52.578742 15012 log.cc:824] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:52.579485 15012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: No bootstrap required, opened a new log
I20260923 15:06:52.580894 15012 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } }
I20260923 15:06:52.581022 15012 raft_consensus.cc:374] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:52.581045 15012 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23945c5404141028d4bca48daaba76b, State: Initialized, Role: FOLLOWER
I20260923 15:06:52.581261 15012 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } }
I20260923 15:06:52.581593 15012 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:52.581593 15017 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } } }
I20260923 15:06:52.581779 15017 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:06:52.583491 15028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:52.583575 15028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:52.586179 14944 master_runner.cc:428] Detected that this master b23945c5404141028d4bca48daaba76b is joining an existing cluster
I20260923 15:06:52.586267 14944 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.250.124:40281
I20260923 15:06:52.594534 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.587510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.7.250.126:38959)
0923 15:06:52.587559 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.587562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.587708 (+ 146us) server_negotiation.cc:407] Connection header received
0923 15:06:52.587860 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.587862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.587876 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.587947 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.588417 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.589220 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590326 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.590635 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590681 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.591714 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.591715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.591716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.591723 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.593280 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.593629 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.593630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.593631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.593689 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.593906 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.593907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.593907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.594042 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.594128 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.594358 (+ 230us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.594436 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:06:52.594547 15034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.587472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:34112)
0923 15:06:52.587642 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.587652 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.587736 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.588001 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.588002 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.588238 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.588327 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.588330 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.589307 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.589308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.590212 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.590214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.590767 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.590827 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.591256 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.591260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.591585 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.593364 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.593365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.593365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.593553 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.593775 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.593776 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.593776 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.593840 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.594119 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.594120 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.594194 (+ 74us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.594328 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.594439 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176011, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.126@KRBTEST.COM for oryx/127.7.250.124@KRBTEST.COM
I20260923 15:06:52.615365 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14344
I20260923 15:06:52.620703 15035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.587742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:48258)
0923 15:06:52.588302 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.588312 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.588373 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.588609 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.588610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.588809 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.588860 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.588861 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.589707 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.589707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.590490 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.590492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.590962 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.591020 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.615240 (+ 24220us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.615246 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.615718 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.617975 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.617976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.617977 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.618783 (+ 806us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.619240 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.619241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.619242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.619298 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.619850 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.619851 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.619907 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.619975 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.620560 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":142,"threads_started":1}
I20260923 15:06:52.621320 15015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.587983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.7.250.124:40281)
0923 15:06:52.588080 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.588082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.588353 (+ 271us) server_negotiation.cc:407] Connection header received
0923 15:06:52.588525 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.588526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.588534 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.588562 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.588935 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.589613 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590568 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.590842 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590878 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.615811 (+ 24933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.615812 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.615813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.615819 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.617841 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.618874 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.618875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.618875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.619167 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.619358 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.619359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.619359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.619502 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.619566 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.620092 (+ 526us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.621215 (+ 1123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1046,"server-negotiator.queue_time_us":69}
I20260923 15:06:52.621644 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:43863
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:38959
--webserver_port=41693
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:06:52.621723 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.588030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52362 (local address 127.7.250.125:33905)
0923 15:06:52.588113 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.588115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.588122 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:52.588143 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.588143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.588152 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.588232 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.588621 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.589375 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590453 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.590755 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590796 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.618660 (+ 27864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.618663 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.618663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.618672 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.620389 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.620809 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.620810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.620811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.620876 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.621094 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.621095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.621095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.621241 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.621287 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.621521 (+ 234us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.621617 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 15:06:52.622058 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.587492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:52362)
0923 15:06:52.587939 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.587949 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.587992 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.588250 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.588251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.588446 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.588520 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.588522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.589474 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.589475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.590353 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.590356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.590877 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.590877 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.590951 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.614988 (+ 24037us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.614994 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.617948 (+ 2954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.620522 (+ 2574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.620523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.620523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.620716 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.620969 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.620970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.620970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.621029 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.621336 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.621337 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.621383 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.621840 (+ 457us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.621952 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":82,"threads_started":1}
I20260923 15:06:52.623080 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.596914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:48274)
0923 15:06:52.596954 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.596964 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.597000 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.597330 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.597331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.597506 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.597551 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.597552 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.598182 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.598184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.599061 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.599064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.599488 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.599489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.599560 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.599983 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.599988 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.601499 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.621864 (+ 20365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.621866 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.621866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.622107 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.622364 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.622365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.622365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.622429 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.622742 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.622743 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.622804 (+ 61us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.622859 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.622988 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 15:06:52.623593 15037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.596995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.7.250.124:40281)
0923 15:06:52.597154 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.597156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.597161 (+ 5us) server_negotiation.cc:407] Connection header received
0923 15:06:52.597187 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.597188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.597197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.597303 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.597632 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.598106 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.599151 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.599413 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.599471 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.601602 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.601603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.601604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.601610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.621752 (+ 20142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.622201 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.622202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.622202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.622281 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.622501 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.622502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.622502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.622630 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.623243 (+ 613us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.623283 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.623368 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":92,"threads_started":1}
I20260923 15:06:52.623999 14282 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.250.124:40281: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b23945c5404141028d4bca48daaba76b"
member_type: NON_VOTER
last_known_addr {
host: "127.7.250.124"
port: 40281
}
attrs {
promote: true
}
}
dest_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1"
cas_config_opid_index: 5
I20260923 15:06:52.624303 14282 consensus_queue.cc:237] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } }
I20260923 15:06:52.624681 14846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b23945c5404141028d4bca48daaba76b (127.7.250.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.625322 14986 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:52.625878 14846 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.250.124:40281
I20260923 15:06:52.625967 14846 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.626027 14846 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:52.626307 14944 master.cc:537] Master@127.7.250.124:40281 shutting down...
I20260923 15:06:52.626530 14986 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Refusing update from remote peer af12aef1b44a4ed8beecad3a7b06b6b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 15:06:52.627079 14845 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:06:52.628716 14267 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260923 15:06:52.630748 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.624873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52378 (local address 127.7.250.125:33905)
0923 15:06:52.627030 (+ 2157us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.627032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.627041 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:52.627066 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.627067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.627093 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.627169 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:52.627367 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.628133 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.629593 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.630224 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.630354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.630441 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.630636 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.630678 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2131}
I20260923 15:06:52.630748 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.624782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:52378)
0923 15:06:52.625044 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.625054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.625088 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.627236 (+ 2148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.627237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.627239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:52.627286 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.627288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.628237 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.628238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.629490 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.629493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.630467 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.630468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.630538 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.630561 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.630615 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.630678 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244}
I20260923 15:06:52.631090 14900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Refusing update from remote peer af12aef1b44a4ed8beecad3a7b06b6b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 15:06:52.631122 14944 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:52.631212 14944 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:52.631258 14944 tablet_replica.cc:330] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: stopping tablet replica
I20260923 15:06:52.631341 14846 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260923 15:06:52.631494 14267 consensus_peers.cc:487] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 -> Peer b23945c5404141028d4bca48daaba76b (127.7.250.124:40281): Couldn't send request to peer b23945c5404141028d4bca48daaba76b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 15:06:52.631772 14900 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b23945c5404141028d4bca48daaba76b (127.7.250.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.633054 14929 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af12aef1b44a4ed8beecad3a7b06b6b1. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.633150 14929 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:06:52.633213 14929 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: NON_VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: true } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.633275 14929 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:06:52.634064 14846 raft_consensus.cc:1050] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: attempting to promote NON_VOTER f0834107645142c39194500863ee9642 to VOTER
I20260923 15:06:52.634310 14846 consensus_queue.cc:237] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } }
I20260923 15:06:52.635049 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.634897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:48288)
0923 15:06:52.634961 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.635015 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:40281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260923 15:06:52.635084 14900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 LEARNER]: Refusing update from remote peer af12aef1b44a4ed8beecad3a7b06b6b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 15:06:52.635291 15042 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260923 15:06:52.635495 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.635408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:48300)
0923 15:06:52.635447 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.635469 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:40281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 15:06:52.635555 14267 consensus_peers.cc:487] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 -> Peer b23945c5404141028d4bca48daaba76b (127.7.250.124:40281): Couldn't send request to peer b23945c5404141028d4bca48daaba76b. Status: Network error: Client connection negotiation failed: client connection to 127.7.250.124:40281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 15:06:52.636055 14846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f0834107645142c39194500863ee9642 (127.7.250.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.636994 14846 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.637084 14846 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:52.637138 14846 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.637188 14846 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:52.637421 14900 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f0834107645142c39194500863ee9642 (127.7.250.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.638634 15041 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.638751 15041 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:52.644605 14944 master.cc:552] Master@127.7.250.124:40281 shutdown complete.
I20260923 15:06:52.644666 14944 master_runner.cc:305] Clearing existing system tablet
I20260923 15:06:52.644826 14944 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.644886 14944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.645707 14944 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:52.646289 15044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.646397 14944 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.646445 14944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:52.646467 14944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
uuid: "b23945c5404141028d4bca48daaba76b"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "f37ba330279185e301522429ce334423"
server_key_iv: "e1434a75b1a84956f4f523f6b1596cd8"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.647215 14944 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:52.648728 14944 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 15:06:52.648794 14944 log.cc:1196] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 15:06:52.648983 14944 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Deleting consensus metadata
I20260923 15:06:52.649153 14944 master_runner.cc:315] Copying system tablet from 127.7.250.126:38959
I20260923 15:06:52.649766 14944 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: tablet copy: Beginning tablet copy session from remote peer at address 127.7.250.126:38959
I20260923 15:06:52.657487 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.650174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.7.250.126:38959)
0923 15:06:52.650226 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.650228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.650601 (+ 373us) server_negotiation.cc:407] Connection header received
0923 15:06:52.650639 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.650640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.650654 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.650721 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.651110 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.651927 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.652992 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.653294 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.653365 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.654492 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.654494 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.654495 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.654503 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.656189 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.656591 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.656593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.656593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.656655 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.656873 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.656875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.656875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.657030 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.657100 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.657267 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.657354 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:06:52.657639 15050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.650091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:34128)
0923 15:06:52.650496 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.650508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.650581 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.650741 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.650741 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.650972 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.651030 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.651033 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.652018 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.652018 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.652899 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.652901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.653379 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.653380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.653443 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.653967 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.653977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.654397 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.656273 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.656274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.656275 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.656511 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.656728 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.656728 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.656729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.656794 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.657103 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.657104 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.657153 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.657226 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.657333 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":71,"threads_started":1}
I20260923 15:06:52.658039 14302 tablet_copy_service.cc:140] P af12aef1b44a4ed8beecad3a7b06b6b1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b23945c5404141028d4bca48daaba76b ({username='oryx', principal='oryx/127.7.250.124@KRBTEST.COM'} at 127.0.0.1:34128)
I20260923 15:06:52.658109 14302 tablet_copy_service.cc:161] P af12aef1b44a4ed8beecad3a7b06b6b1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b23945c5404141028d4bca48daaba76b at {username='oryx', principal='oryx/127.7.250.124@KRBTEST.COM'} at 127.0.0.1:34128: session id = b23945c5404141028d4bca48daaba76b-00000000000000000000000000000000
I20260923 15:06:52.658756 14302 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 15:06:52.659276 14944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 15:06:52.660563 14944 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: tablet copy: Starting download of 0 data blocks...
I20260923 15:06:52.660712 14944 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: tablet copy: Starting download of 1 WAL segments...
I20260923 15:06:52.661442 14944 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 15:06:52.662354 14302 tablet_copy_service.cc:342] P af12aef1b44a4ed8beecad3a7b06b6b1: Request end of tablet copy session b23945c5404141028d4bca48daaba76b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.250.124@KRBTEST.COM'} at 127.0.0.1:34128
I20260923 15:06:52.662410 14302 tablet_copy_service.cc:433] P af12aef1b44a4ed8beecad3a7b06b6b1: ending tablet copy session b23945c5404141028d4bca48daaba76b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b23945c5404141028d4bca48daaba76b
I20260923 15:06:52.665052 14944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:52.666318 15054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.666359 15051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.666950 15052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.674505 14944 server_base.cc:1034] running on GCE node
I20260923 15:06:52.674629 14944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.674799 14944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.675948 14944 hybrid_clock.cc:648] HybridClock initialized: now 1790176012675940 us; error 31 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.677462 14944 init.cc:377] Logged in from keytab as oryx/127.7.250.124@KRBTEST.COM (short username oryx)
I20260923 15:06:52.677734 14944 webserver.cc:460] Webserver started at http://127.7.250.124:34637/ using document root <none> and password file <none>
I20260923 15:06:52.677850 14944 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.677878 14944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.682540 14944 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.001s
I20260923 15:06:52.683187 15061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.683332 14944 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.683373 14944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
uuid: "b23945c5404141028d4bca48daaba76b"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "f37ba330279185e301522429ce334423"
server_key_iv: "e1434a75b1a84956f4f523f6b1596cd8"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.683435 14944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.701881 14944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.703923 14944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.706861 14944 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.124:40281
I20260923 15:06:52.707262 14944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
I20260923 15:06:52.708418 15114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.124:40281 every 8 connection(s)
I20260923 15:06:52.709405 15115 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:06:52.709985 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Bootstrap starting.
I20260923 15:06:52.713047 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:06:52.713387 15115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: Bootstrap complete.
I20260923 15:06:52.713640 15115 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } }
I20260923 15:06:52.713822 15115 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b23945c5404141028d4bca48daaba76b, State: Initialized, Role: LEARNER
I20260923 15:06:52.713910 15115 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } }
I20260923 15:06:52.714198 15115 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:52.717105 15129 catalog_manager.cc:1259] Loaded cluster ID: c0ae394f1adb4c1b9dd72b947907434a
I20260923 15:06:52.717203 15129 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: loading cluster ID for follower catalog manager: success
I20260923 15:06:52.718384 15129 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: acquiring CA information for follower catalog manager: success
I20260923 15:06:52.718801 15129 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:06:52.718982 15118 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:52.719141 15118 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: This master's current role is: LEARNER
W20260923 15:06:52.753212 15038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:52.753542 15038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:52.753590 15038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:52.756218 15038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:52.756338 15038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:52.756413 15038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:52.759648 15038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:43863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=41693
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:52.759976 15038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:52.760303 15038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:52.766842 15134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.766873 15137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.766976 15135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.767091 15038 server_base.cc:1034] running on GCE node
I20260923 15:06:52.767663 15038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.768018 15038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.769194 15038 hybrid_clock.cc:648] HybridClock initialized: now 1790176012769166 us; error 48 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.772167 15038 init.cc:377] Logged in from keytab as oryx/127.7.250.65@KRBTEST.COM (short username oryx)
I20260923 15:06:52.772646 15038 webserver.cc:460] Webserver started at http://127.7.250.65:41693/ using document root <none> and password file <none>
I20260923 15:06:52.772886 15038 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.772964 15038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.774441 15038 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:06:52.775100 15144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.775273 15038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:52.775380 15038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "c412d6a53bf84c72a8433a17e118bd55"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "6eece13df4cd2cf3b4083adbe66c8616"
server_key_iv: "02720f9b8177f5b41a50c94161c2924e"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.775739 15038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.793002 15038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.795439 15038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:52.795614 15038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.795876 15038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:52.796204 15038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:52.796262 15038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.796303 15038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:52.796335 15038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.802532 15038 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43863
I20260923 15:06:52.802625 15258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43863 every 8 connection(s)
I20260923 15:06:52.802968 15038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:52.804291 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 15038
I20260923 15:06:52.804486 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14479
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:52.811249 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:35141
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:38959
--webserver_port=34663
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.124@KRBTEST.COM
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
I20260923 15:06:52.816062 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.803750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.7.250.65:45125)
0923 15:06:52.804504 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.804516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.805627 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.805884 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.805885 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.806236 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.806512 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.806520 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.808007 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.808008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.809138 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.809140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.809887 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.809888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.809973 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.810461 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.810468 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.811967 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.814076 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.814084 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.814086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.814469 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.814688 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.814688 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.814689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.814881 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.815315 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.815321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.815482 (+ 161us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.815599 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.815786 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":2,"spinlock_wait_cycles":7936,"thread_start_us":114,"threads_started":1}
I20260923 15:06:52.816516 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.803745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:60375 (local address 127.7.250.126:38959)
0923 15:06:52.804076 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.804079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.804094 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:06:52.805425 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.805427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.805447 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.805479 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.806645 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.807297 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.808331 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.808542 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.808630 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.811846 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.811848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.811848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.811858 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.814466 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.814897 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.814898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.814899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.814960 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.815184 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.815185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.815185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.815657 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.815715 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.816269 (+ 554us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.816369 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301}
I20260923 15:06:52.816987 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.803749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:60375)
0923 15:06:52.803958 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.804003 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.805293 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.805671 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.805675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.806150 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.806513 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.806519 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.807409 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.807410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.808236 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.808238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.808642 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.808643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.808778 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.809722 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.809732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.811725 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.814561 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.814562 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.814563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.814808 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.815035 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.815036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.815036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.815099 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.815775 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.815776 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.815831 (+ 55us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.816667 (+ 836us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.816797 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":36,"thread_start_us":127,"threads_started":1}
I20260923 15:06:52.817010 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.804075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:45125 (local address 127.7.250.124:40281)
0923 15:06:52.804263 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.804266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.805566 (+ 1300us) server_negotiation.cc:407] Connection header received
0923 15:06:52.805727 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.805729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.805744 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.805791 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.806628 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.807897 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.809227 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.809531 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.809597 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.812053 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.812054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.812055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.812062 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.813954 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.814555 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.814556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.814557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.814615 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.815033 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.815047 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.815048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.815219 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.815278 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.815966 (+ 688us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.816047 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":60,"threads_started":1}
I20260923 15:06:52.817437 15259 heartbeater.cc:347] Connected to a master server at 127.7.250.124:40281
I20260923 15:06:52.817597 15259 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.817813 15260 heartbeater.cc:347] Connected to a master server at 127.7.250.126:38959
I20260923 15:06:52.817945 15260 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.818048 15260 heartbeater.cc:511] Master 127.7.250.126:38959 requested a full tablet report, sending...
I20260923 15:06:52.818118 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.804192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:44385 (local address 127.7.250.125:33905)
0923 15:06:52.804258 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.804260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.806080 (+ 1820us) server_negotiation.cc:407] Connection header received
0923 15:06:52.809311 (+ 3231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.809313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.809325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.809471 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.809858 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.810915 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.811957 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.812718 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.812767 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.815110 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.815112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.815112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.815120 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.816844 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.817206 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.817207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.817207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.817267 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.817513 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.817514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.817515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.817675 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.817733 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.817944 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.818023 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:06:52.817848 15259 heartbeater.cc:511] Master 127.7.250.124:40281 requested a full tablet report, sending...
I20260923 15:06:52.818578 14282 ts_manager.cc:194] Re-registered known tserver with Master: c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863)
I20260923 15:06:52.818660 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.803750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.7.250.65:44385)
0923 15:06:52.806020 (+ 2270us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.806031 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.809249 (+ 3218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.809520 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.809521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.809712 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.809757 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.809760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.811011 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.811012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.811856 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.811859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.812853 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.812854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.812908 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.813483 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.813488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.815021 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.816910 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.816911 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.816911 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.817117 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.817374 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.817375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.817375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.817437 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.817782 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.817782 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.817822 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.818198 (+ 376us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.818333 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2242,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260923 15:06:52.819010 14282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:60375
I20260923 15:06:52.819033 15070 ts_manager.cc:194] Registered new tserver with Master: c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863)
I20260923 15:06:52.819351 15261 heartbeater.cc:347] Connected to a master server at 127.7.250.125:33905
I20260923 15:06:52.819402 15261 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.819468 15261 heartbeater.cc:511] Master 127.7.250.125:33905 requested a full tablet report, sending...
I20260923 15:06:52.819878 14890 ts_manager.cc:194] Registered new tserver with Master: c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863)
W20260923 15:06:52.898937 15267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:52.899144 15267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:52.899168 15267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:52.900411 15267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:52.900461 15267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:52.900489 15267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:52.902038 15267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:35141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=34663
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:52.902328 15267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:52.902601 15267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:52.907857 15272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.907829 15273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:52.907852 15275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:52.908138 15267 server_base.cc:1034] running on GCE node
I20260923 15:06:52.908357 15267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:52.908629 15267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:52.909785 15267 hybrid_clock.cc:648] HybridClock initialized: now 1790176012909762 us; error 39 us; skew 500 ppm
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:52.912467 15267 init.cc:377] Logged in from keytab as oryx/127.7.250.66@KRBTEST.COM (short username oryx)
I20260923 15:06:52.912955 15267 webserver.cc:460] Webserver started at http://127.7.250.66:34663/ using document root <none> and password file <none>
I20260923 15:06:52.913225 15267 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:52.913317 15267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:52.915375 15267 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:52.916114 15282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.916388 15267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:52.916497 15267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "e72bdea6a7684fde9af7a8c65f28ed01"
format_stamp: "Formatted at 2026-09-23 15:06:51 on dist-test-slave-bb3x"
server_key: "55c05b6fbf5ad2623600e030148a58b7"
server_key_iv: "6b2f629e7fd4f0090319d0bd115c88d3"
server_key_version: "encryptionkey@0"
I20260923 15:06:52.916889 15267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:52.929816 15267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:52.932127 15267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:52.932345 15267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:52.932598 15267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:52.932920 15267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:52.932979 15267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.933017 15267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:52.933043 15267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:52.939088 15267 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:35141
I20260923 15:06:52.939163 15396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:35141 every 8 connection(s)
I20260923 15:06:52.939543 15267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.124@KRBTEST.COM
I20260923 15:06:52.947670 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 15267
I20260923 15:06:52.947855 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14614
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
Sep 23 15:06:52 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176012, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:52.951406 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.940435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:36705 (local address 127.7.250.124:40281)
0923 15:06:52.940495 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.940497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.941538 (+ 1041us) server_negotiation.cc:407] Connection header received
0923 15:06:52.941572 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.941573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.941590 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.941661 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.942357 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.943143 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944502 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.944758 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944808 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.947537 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.947539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.947539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.947547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.949676 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.950199 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.950200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.950200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.950277 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.950566 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.950567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.950568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.950719 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.950765 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.951198 (+ 433us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.951300 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:06:52.951992 15403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.940346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.7.250.66:36705)
0923 15:06:52.941468 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.941477 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.941523 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.941671 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.941672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.942025 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.942250 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.942273 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.943232 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.943233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.944411 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.944414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.944838 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.944911 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.945430 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.945435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.947419 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.949775 (+ 2356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.949783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.949785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.950091 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.950354 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.950354 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.950355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.950483 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.950814 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.950821 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.951001 (+ 180us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.951497 (+ 496us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.951719 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":123,"thread_start_us":89,"threads_started":1}
I20260923 15:06:52.953979 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.940649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:49413 (local address 127.7.250.125:33905)
0923 15:06:52.940713 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.940716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.940830 (+ 114us) server_negotiation.cc:407] Connection header received
0923 15:06:52.941271 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.941272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.941286 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.941340 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.942475 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.943222 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944252 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.944485 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944558 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.951231 (+ 6673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.951232 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.951233 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.951240 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.952714 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.953093 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.953094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.953094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.953155 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.953365 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.953365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.953366 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.953522 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.953590 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.953784 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.953859 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:06:52.954314 15401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.940340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.7.250.66:49413)
0923 15:06:52.940753 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.940763 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.941164 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.942014 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.942015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.942161 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.942250 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.942256 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.943289 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.943290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.944153 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.944155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.944574 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.944678 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.945276 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.945288 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.950208 (+ 4920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.952832 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.952833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.952834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.953003 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.953234 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.953234 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.953235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.953297 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.953618 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.953618 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.953663 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.954063 (+ 400us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.954183 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":82,"thread_start_us":107,"threads_started":1}
I20260923 15:06:52.954628 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.940340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:52697)
0923 15:06:52.940574 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:52.940618 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:52.941147 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:52.941416 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:52.941421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:52.942027 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:52.942250 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.942301 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.943121 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.943122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.944411 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:52.944414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.945240 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.945241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:52.945387 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.945801 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:52.945806 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:52.950587 (+ 4781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:52.953469 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.953470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.953470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.953683 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.953915 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:52.953915 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:52.953916 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:52.953972 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:52.954307 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:52.954308 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:52.954353 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 15:06:52.954425 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:06:52.954524 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":83,"thread_start_us":94,"threads_started":1}
I20260923 15:06:52.954649 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:52.940583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:52697 (local address 127.7.250.126:38959)
0923 15:06:52.940644 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:52.940646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:52.940696 (+ 50us) server_negotiation.cc:407] Connection header received
0923 15:06:52.941271 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:52.941272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:52.941286 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:52.941345 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:52.942437 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.942997 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.944854 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:52.945152 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:52.945230 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:52.950697 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:52.950698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:52.950699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:52.950707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:52.953387 (+ 2680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.953770 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.953771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.953771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.953839 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:52.954049 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:52.954050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:52.954050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:52.954210 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:52.954309 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:52.954471 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 15:06:52.954550 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:06:52.954700 15399 heartbeater.cc:347] Connected to a master server at 127.7.250.125:33905
I20260923 15:06:52.954815 15399 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.955029 15399 heartbeater.cc:511] Master 127.7.250.125:33905 requested a full tablet report, sending...
I20260923 15:06:52.955175 15398 heartbeater.cc:347] Connected to a master server at 127.7.250.126:38959
I20260923 15:06:52.955240 15398 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.955308 15398 heartbeater.cc:511] Master 127.7.250.126:38959 requested a full tablet report, sending...
I20260923 15:06:52.955452 15397 heartbeater.cc:347] Connected to a master server at 127.7.250.124:40281
I20260923 15:06:52.955498 15397 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:52.955539 14890 ts_manager.cc:194] Registered new tserver with Master: e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:52.955554 14282 ts_manager.cc:194] Re-registered known tserver with Master: e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:52.955562 15397 heartbeater.cc:511] Master 127.7.250.124:40281 requested a full tablet report, sending...
I20260923 15:06:52.955890 15070 ts_manager.cc:194] Registered new tserver with Master: e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:52.956027 14282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:52697
I20260923 15:06:52.958557 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:42125
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:38959
--webserver_port=46481
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:53.042582 15404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:53.042821 15404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:53.042861 15404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:53.044039 15404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:53.044106 15404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:53.044165 15404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:53.045743 15404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:42125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=46481
--tserver_master_addrs=127.7.250.126:38959,127.7.250.125:33905,127.7.250.124:40281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:53.046015 15404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:53.046288 15404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:53.051429 15412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:53.051534 15409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:53.051626 15404 server_base.cc:1034] running on GCE node
W20260923 15:06:53.051754 15410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:53.051966 15404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:53.052243 15404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:53.053411 15404 hybrid_clock.cc:648] HybridClock initialized: now 1790176013053397 us; error 51 us; skew 500 ppm
Sep 23 15:06:53 dist-test-slave-bb3x krb5kdc[14237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:53.055965 15404 init.cc:377] Logged in from keytab as oryx/127.7.250.67@KRBTEST.COM (short username oryx)
I20260923 15:06:53.056427 15404 webserver.cc:460] Webserver started at http://127.7.250.67:46481/ using document root <none> and password file <none>
I20260923 15:06:53.056694 15404 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:53.056798 15404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:53.058382 15404 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:06:53.059056 15419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:53.059422 15404 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:53.059530 15404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "cf6d0e8ef4a746599484cb9dc4f217cd"
format_stamp: "Formatted at 2026-09-23 15:06:52 on dist-test-slave-bb3x"
server_key: "697001d5c6337be57babb8b51844e135"
server_key_iv: "6a82a96227b5183e6627410fa4720629"
server_key_version: "encryptionkey@0"
I20260923 15:06:53.059875 15404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:53.081187 15404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:53.083274 15404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:53.083494 15404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:53.083745 15404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:06:53.084065 15404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:53.084121 15404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:53.084164 15404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:53.084192 15404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:53.090065 15404 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:42125
I20260923 15:06:53.090126 15533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:42125 every 8 connection(s)
I20260923 15:06:53.090575 15404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:53.094307 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 15404
Sep 23 15:06:53 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
Sep 23 15:06:53 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.124@KRBTEST.COM
Sep 23 15:06:53 dist-test-slave-bb3x krb5kdc[14237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176013, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.125@KRBTEST.COM
I20260923 15:06:53.112847 15539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.091713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.7.250.67:38549)
0923 15:06:53.100074 (+ 8361us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.100084 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.100137 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.100745 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.100746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.100983 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:53.101084 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.101086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.102033 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.102033 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.103093 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.103096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.105524 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.105525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.105603 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.106100 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:53.106105 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:53.107734 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:53.110918 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:53.110931 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:53.110934 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:53.111291 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:53.111519 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:53.111520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:53.111521 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:53.111683 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:53.112016 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:53.112025 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:53.112216 (+ 191us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.112325 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.112527 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8339,"mutex_wait_us":65,"thread_start_us":122,"threads_started":1}
I20260923 15:06:53.113070 15540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.091693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:48489)
0923 15:06:53.092429 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.092504 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.093778 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.095883 (+ 2105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.095888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.096734 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:53.097237 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.097243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.098244 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.098244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.099951 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.099954 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.100444 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.100445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.100598 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.103580 (+ 2982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:53.103590 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:53.105652 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:53.111936 (+ 6284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:53.111937 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:53.111938 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:53.112149 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:53.112380 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:53.112381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:53.112381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:53.112444 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:53.112738 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:53.112739 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:53.112790 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.112845 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.112947 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":207,"threads_started":1}
I20260923 15:06:53.113488 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.091853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:48489 (local address 127.7.250.126:38959)
0923 15:06:53.091981 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.091983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.093038 (+ 1055us) server_negotiation.cc:407] Connection header received
0923 15:06:53.093894 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.093897 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.093912 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.093989 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.097359 (+ 3370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.098136 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.100041 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.100359 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.100435 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.105797 (+ 5362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.105799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.105799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.105808 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.111867 (+ 6059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.112245 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.112247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.112247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.112307 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.112512 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.112512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.112513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.112648 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.112710 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.113289 (+ 579us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.113376 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260923 15:06:53.113791 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.091690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:38549 (local address 127.7.250.124:40281)
0923 15:06:53.091744 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.091746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.100309 (+ 8563us) server_negotiation.cc:407] Connection header received
0923 15:06:53.100336 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.100337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.100349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.100411 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.101211 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.101938 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.105080 (+ 3142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.105371 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.105463 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.107862 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.107864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.107864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.107872 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.110846 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.111373 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.111374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.111374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.111439 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.111764 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.111765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.111765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.111913 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.111987 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.113027 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.113666 (+ 639us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:06:53.114444 15534 heartbeater.cc:347] Connected to a master server at 127.7.250.124:40281
I20260923 15:06:53.114552 15534 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:53.114782 15534 heartbeater.cc:511] Master 127.7.250.124:40281 requested a full tablet report, sending...
I20260923 15:06:53.115373 15535 heartbeater.cc:347] Connected to a master server at 127.7.250.126:38959
I20260923 15:06:53.115425 15535 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:53.115504 15535 heartbeater.cc:511] Master 127.7.250.126:38959 requested a full tablet report, sending...
I20260923 15:06:53.115526 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.095827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.7.250.124:40281)
0923 15:06:53.096266 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.096269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.096418 (+ 149us) server_negotiation.cc:407] Connection header received
0923 15:06:53.096445 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.096446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.096458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.096519 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.097341 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.098211 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.099545 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.100050 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.100117 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.105719 (+ 5602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.105721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.105721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.105730 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.114010 (+ 8280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.114445 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.114446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.114447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.114507 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.114703 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.114704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.114704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.114863 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.114975 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.115286 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.115378 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":43,"threads_started":1}
I20260923 15:06:53.115801 15070 ts_manager.cc:194] Registered new tserver with Master: cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125)
I20260923 15:06:53.115995 14282 ts_manager.cc:194] Re-registered known tserver with Master: cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125)
I20260923 15:06:53.116117 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.100509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34142 (local address 127.7.250.126:38959)
0923 15:06:53.100679 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.100681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.100697 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:06:53.100719 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.100720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.100733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.101329 (+ 596us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.101499 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.102238 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.105101 (+ 2863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.105624 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.106501 (+ 877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.110653 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.110725 (+ 72us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.110726 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.110742 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.114328 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.115135 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.115136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.115137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.115200 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.115409 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.115411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.115411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.115547 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.115630 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.115884 (+ 254us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.115974 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":103,"threads_started":1}
I20260923 15:06:53.116729 14282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:48489
I20260923 15:06:53.117445 15548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.095873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.7.250.125:33905)
0923 15:06:53.100235 (+ 4362us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.100237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.100244 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:53.100267 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.100268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.100280 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.100340 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.101644 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.102354 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.104664 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.104983 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.105044 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.110823 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.110824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.110825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.110833 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.114537 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.116088 (+ 1551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.116090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.116090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.116150 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.116573 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.116574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.116575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.116712 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.116788 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.117209 (+ 421us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.117314 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4339,"thread_start_us":54,"threads_started":1}
I20260923 15:06:53.118062 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.092098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:46045 (local address 127.7.250.125:33905)
0923 15:06:53.092148 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.092150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.100889 (+ 8739us) server_negotiation.cc:407] Connection header received
0923 15:06:53.100992 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.100993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.101004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.101055 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.101560 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.102383 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.105959 (+ 3576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.106326 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.106366 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.113005 (+ 6639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.113006 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.113006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.113014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.115882 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.116513 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.116514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.116515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.116566 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.117074 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.117075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.117076 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.117210 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.117250 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.117897 (+ 647us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.117974 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":25}
I20260923 15:06:53.118366 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.091995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.7.250.67:46045)
0923 15:06:53.100827 (+ 8832us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.100837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.100915 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.101241 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.101242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.101409 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:53.101470 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.101472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.102613 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.102614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.105872 (+ 3258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.105874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.106447 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.106448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.106516 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.109407 (+ 2891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:53.109412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:53.112910 (+ 3498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:53.116215 (+ 3305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:53.116216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:53.116217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:53.116425 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:53.116639 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:53.116640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:53.116640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:53.116690 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:53.117461 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:53.117462 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:53.117508 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.117871 (+ 363us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.117983 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8813,"mutex_wait_us":11,"thread_start_us":100,"threads_started":1}
I20260923 15:06:53.118757 15536 heartbeater.cc:347] Connected to a master server at 127.7.250.125:33905
I20260923 15:06:53.118811 15536 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:53.118872 15536 heartbeater.cc:511] Master 127.7.250.125:33905 requested a full tablet report, sending...
I20260923 15:06:53.119141 14890 ts_manager.cc:194] Registered new tserver with Master: cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125)
I20260923 15:06:53.117959 14282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:53.120471 14282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:53.128782 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.124270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43863 (local address 127.0.0.1:50884)
0923 15:06:53.124408 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.124419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.124461 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.125012 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.125013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.125015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:53.125066 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.125068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.126481 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.126482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.127605 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.127607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.128571 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.128572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.128631 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.128635 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.128663 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.128701 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":138,"thread_start_us":95,"threads_started":1}
I20260923 15:06:53.129143 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.124600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50884 (local address 127.7.250.65:43863)
0923 15:06:53.124774 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.124779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.124789 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:53.124896 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.124899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.124917 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.125002 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:53.125146 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.126401 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.127694 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.128469 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.128903 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.129002 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.129030 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.129068 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":91,"threads_started":1}
I20260923 15:06:53.129269 15556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.124288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:35141 (local address 127.0.0.1:49044)
0923 15:06:53.124859 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.124868 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.124927 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.125282 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.125283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.125285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:53.125324 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.125325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.127043 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.127044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.127935 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.127937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.129004 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.129005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.129077 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.129083 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.129135 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.129188 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":90,"threads_started":1}
I20260923 15:06:53.129714 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.124678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.7.250.66:35141)
0923 15:06:53.124832 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.124837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.124933 (+ 96us) server_negotiation.cc:407] Connection header received
0923 15:06:53.125186 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.125189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.125205 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.125284 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:53.125383 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.126968 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.128022 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.128892 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.129496 (+ 604us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.129585 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.129618 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.129650 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260923 15:06:53.130491 15193 tablet_service.cc:1467] Processing CreateTablet for tablet 482e2bab9f5540ad849a592540338192 (DEFAULT_TABLE table=test-table [id=56973d3331794b449a9efa05bc4fe8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:53.130483 15331 tablet_service.cc:1467] Processing CreateTablet for tablet 482e2bab9f5540ad849a592540338192 (DEFAULT_TABLE table=test-table [id=56973d3331794b449a9efa05bc4fe8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:53.130879 15193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482e2bab9f5540ad849a592540338192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:53.130888 15331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482e2bab9f5540ad849a592540338192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:53.133128 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.124231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42125 (local address 127.0.0.1:36606)
0923 15:06:53.124408 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.124419 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.124474 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.125400 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.125401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.125402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:53.125447 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.125447 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.126722 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.126723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.129451 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.129454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.130674 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.130675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.130748 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.130763 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.130815 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.130857 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260923 15:06:53.133628 15557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.124326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36606 (local address 127.7.250.67:42125)
0923 15:06:53.125076 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.125080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.125090 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:53.125126 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.125129 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.125145 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.125217 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:53.125524 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.126567 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.129546 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.130492 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.130562 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.130638 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.133321 (+ 2683us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.133540 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":78,"threads_started":1}
I20260923 15:06:53.134328 15468 tablet_service.cc:1467] Processing CreateTablet for tablet 482e2bab9f5540ad849a592540338192 (DEFAULT_TABLE table=test-table [id=56973d3331794b449a9efa05bc4fe8d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:53.134693 15468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 482e2bab9f5540ad849a592540338192. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:53.134893 15561 tablet_bootstrap.cc:492] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Bootstrap starting.
I20260923 15:06:53.135551 15561 tablet_bootstrap.cc:654] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:53.135867 15561 log.cc:824] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:53.136749 15561 tablet_bootstrap.cc:492] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: No bootstrap required, opened a new log
I20260923 15:06:53.136809 15561 ts_tablet_manager.cc:1397] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:53.139302 15563 tablet_bootstrap.cc:492] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Bootstrap starting.
I20260923 15:06:53.139576 15560 tablet_bootstrap.cc:492] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Bootstrap starting.
I20260923 15:06:53.139509 15561 raft_consensus.cc:348] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.139694 15561 raft_consensus.cc:374] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:53.139729 15561 raft_consensus.cc:729] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c412d6a53bf84c72a8433a17e118bd55, State: Initialized, Role: FOLLOWER
I20260923 15:06:53.139894 15561 consensus_queue.cc:260] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.140165 15561 ts_tablet_manager.cc:1428] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:06:53.140254 15560 tablet_bootstrap.cc:654] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:53.140276 15260 heartbeater.cc:503] Master 127.7.250.126:38959 was elected leader, sending a full tablet report...
I20260923 15:06:53.140367 15563 tablet_bootstrap.cc:654] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:53.140574 15560 log.cc:824] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:53.140691 15563 log.cc:824] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:53.141543 15560 tablet_bootstrap.cc:492] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: No bootstrap required, opened a new log
I20260923 15:06:53.141620 15560 ts_tablet_manager.cc:1397] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:53.142828 15563 tablet_bootstrap.cc:492] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: No bootstrap required, opened a new log
I20260923 15:06:53.142920 15563 ts_tablet_manager.cc:1397] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 15:06:53.143864 15560 raft_consensus.cc:348] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.143996 15560 raft_consensus.cc:374] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:53.144017 15560 raft_consensus.cc:729] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72bdea6a7684fde9af7a8c65f28ed01, State: Initialized, Role: FOLLOWER
I20260923 15:06:53.144151 15560 consensus_queue.cc:260] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.144423 15560 ts_tablet_manager.cc:1428] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:53.144502 15398 heartbeater.cc:503] Master 127.7.250.126:38959 was elected leader, sending a full tablet report...
I20260923 15:06:53.145563 15563 raft_consensus.cc:348] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.145714 15563 raft_consensus.cc:374] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:53.145761 15563 raft_consensus.cc:729] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf6d0e8ef4a746599484cb9dc4f217cd, State: Initialized, Role: FOLLOWER
I20260923 15:06:53.145888 15563 consensus_queue.cc:260] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.146131 15563 ts_tablet_manager.cc:1428] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:53.146183 15535 heartbeater.cc:503] Master 127.7.250.126:38959 was elected leader, sending a full tablet report...
I20260923 15:06:53.164029 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.161131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:48310)
0923 15:06:53.161190 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.161204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.161263 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.161523 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.161524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.161526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:53.161576 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.161577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.162339 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.162340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.163091 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.163093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.163740 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.163741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.163804 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.163808 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.163902 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.163952 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:06:53.164029 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.161282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.7.250.124:40281)
0923 15:06:53.161343 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.161345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.161352 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:53.161378 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.161378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.161400 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.161455 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:53.161645 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.162235 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.163185 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.163631 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.163738 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.163818 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.163911 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.163957 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:06:53.164458 15089 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260923 15:06:53.165230 15118 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: SysCatalogTable state changed. Reason: New leader af12aef1b44a4ed8beecad3a7b06b6b1. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:53.165357 15118 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: This master's current role is: LEARNER
W20260923 15:06:53.191650 15400 tablet.cc:2369] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:53.304801 15262 tablet.cc:2369] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:53.342572 15537 tablet.cc:2369] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:53.445181 15568 raft_consensus.cc:482] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:53.445370 15568 raft_consensus.cc:504] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.445760 15568 leader_election.cc:288] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863), e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:53.448941 15540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.446012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:35141 (local address 127.7.250.67:57131)
0923 15:06:53.446192 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.446203 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.446271 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.446452 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.446453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.446455 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:53.446511 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.446512 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.447232 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.447233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.447970 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.447972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.448654 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.448656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.448740 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.448743 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.448804 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.448849 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":136}
I20260923 15:06:53.448972 15559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.446121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:57131 (local address 127.7.250.66:35141)
0923 15:06:53.446274 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.446276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.446310 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:06:53.446367 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.446368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.446379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.446435 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:53.446606 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.447141 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.448062 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.448466 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.448662 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.448678 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.448822 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.448908 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260923 15:06:53.448951 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.446159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:32981 (local address 127.7.250.65:43863)
0923 15:06:53.446225 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.446227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.446237 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:53.446294 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.446295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.446306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.446365 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:53.446572 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.447070 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.448169 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.448562 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.448658 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.448668 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.448835 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.448874 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:06:53.449244 15538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.446008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43863 (local address 127.7.250.67:32981)
0923 15:06:53.446097 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:53.446112 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:53.446186 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:53.446386 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:53.446387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:53.446389 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:53.446485 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.446487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.447160 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.447161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.448058 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:53.448061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.448661 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.448662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:53.448754 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.448756 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:53.448830 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:06:53.448874 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 15:06:53.449381 15351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482e2bab9f5540ad849a592540338192" candidate_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" is_pre_election: true
I20260923 15:06:53.449573 15351 raft_consensus.cc:2454] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf6d0e8ef4a746599484cb9dc4f217cd in term 0.
I20260923 15:06:53.449618 15213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482e2bab9f5540ad849a592540338192" candidate_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c412d6a53bf84c72a8433a17e118bd55" is_pre_election: true
I20260923 15:06:53.449759 15213 raft_consensus.cc:2454] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf6d0e8ef4a746599484cb9dc4f217cd in term 0.
I20260923 15:06:53.449810 15420 leader_election.cc:302] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf6d0e8ef4a746599484cb9dc4f217cd, e72bdea6a7684fde9af7a8c65f28ed01; no voters:
I20260923 15:06:53.449942 15568 raft_consensus.cc:2781] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:53.450007 15568 raft_consensus.cc:482] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:53.450052 15568 raft_consensus.cc:3037] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:53.450853 15568 raft_consensus.cc:504] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.451009 15568 leader_election.cc:288] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [CANDIDATE]: Term 1 election: Requested vote from peers c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863), e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:53.451131 15213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482e2bab9f5540ad849a592540338192" candidate_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c412d6a53bf84c72a8433a17e118bd55"
I20260923 15:06:53.451190 15351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "482e2bab9f5540ad849a592540338192" candidate_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72bdea6a7684fde9af7a8c65f28ed01"
I20260923 15:06:53.451205 15213 raft_consensus.cc:3037] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:53.451264 15351 raft_consensus.cc:3037] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:53.452076 15213 raft_consensus.cc:2454] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf6d0e8ef4a746599484cb9dc4f217cd in term 1.
I20260923 15:06:53.452076 15351 raft_consensus.cc:2454] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf6d0e8ef4a746599484cb9dc4f217cd in term 1.
I20260923 15:06:53.452401 15420 leader_election.cc:302] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf6d0e8ef4a746599484cb9dc4f217cd, e72bdea6a7684fde9af7a8c65f28ed01; no voters:
I20260923 15:06:53.452509 15568 raft_consensus.cc:2781] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:53.452649 15568 raft_consensus.cc:686] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 LEADER]: Becoming Leader. State: Replica: cf6d0e8ef4a746599484cb9dc4f217cd, State: Running, Role: LEADER
I20260923 15:06:53.452772 15568 consensus_queue.cc:237] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:53.453573 14282 catalog_manager.cc:5495] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd reported cstate change: term changed from 0 to 1, leader changed from <none> to cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67). New cstate: current_term: 1 leader_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:53.454159 14846 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: NON_VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: true } } }
I20260923 15:06:53.454249 14846 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:53.455101 15042 raft_consensus.cc:1050] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: attempting to promote NON_VOTER b23945c5404141028d4bca48daaba76b to VOTER
I20260923 15:06:53.455353 15042 consensus_queue.cc:237] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } }
I20260923 15:06:53.455862 14900 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Refusing update from remote peer af12aef1b44a4ed8beecad3a7b06b6b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260923 15:06:53.455935 15088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 LEARNER]: Refusing update from remote peer af12aef1b44a4ed8beecad3a7b06b6b1: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260923 15:06:53.456110 14845 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260923 15:06:53.456267 15574 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260923 15:06:53.456888 15574 raft_consensus.cc:2932] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b23945c5404141028d4bca48daaba76b (127.7.250.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.457018 14900 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b23945c5404141028d4bca48daaba76b (127.7.250.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.457048 15088 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, b23945c5404141028d4bca48daaba76b (127.7.250.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.457839 15574 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.457924 15574 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:53.457993 15574 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.458043 15574 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:53.457988 15118 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.458071 15118 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:53.458029 14846 sys_catalog.cc:455] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.458003 15576 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:53.458101 14846 sys_catalog.cc:458] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:53.458106 15576 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:53.481410 15557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.478969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.7.250.67:42125)
0923 15:06:53.479029 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.479030 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.479039 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:53.479076 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.479077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.479094 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.479168 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:53.479296 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.479859 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.480641 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.480843 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.480905 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.481086 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:53.481216 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:53.481308 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.481325 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.481347 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:06:53.483496 15351 raft_consensus.cc:1261] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Refusing update from remote peer cf6d0e8ef4a746599484cb9dc4f217cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:53.483541 15213 raft_consensus.cc:1261] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 1 FOLLOWER]: Refusing update from remote peer cf6d0e8ef4a746599484cb9dc4f217cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:53.483904 15568 consensus_queue.cc:1035] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:53.484061 15572 consensus_queue.cc:1035] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:53.486640 15578 mvcc.cc:205] Tried to move back new op lower bound from 7332560951225450496 to 7332560951103029248. Current Snapshot: MvccSnapshot[applied={T|T < 7332560951225450496}]
I20260923 15:06:53.486796 15580 mvcc.cc:205] Tried to move back new op lower bound from 7332560951225450496 to 7332560951103029248. Current Snapshot: MvccSnapshot[applied={T|T < 7332560951225450496}]
I20260923 15:06:53.487993 15579 mvcc.cc:205] Tried to move back new op lower bound from 7332560951225450496 to 7332560951103029248. Current Snapshot: MvccSnapshot[applied={T|T < 7332560951225450496}]
I20260923 15:06:53.489698 14281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:53.489840 14281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:53.491266 14281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af12aef1b44a4ed8beecad3a7b06b6b1: Sending DeleteTablet for 3 replicas of tablet 482e2bab9f5540ad849a592540338192
I20260923 15:06:53.491641 15193 tablet_service.cc:1514] Processing DeleteTablet for tablet 482e2bab9f5540ad849a592540338192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:53 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:50884
I20260923 15:06:53.491688 15331 tablet_service.cc:1514] Processing DeleteTablet for tablet 482e2bab9f5540ad849a592540338192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:53 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:49044
I20260923 15:06:53.491798 15468 tablet_service.cc:1514] Processing DeleteTablet for tablet 482e2bab9f5540ad849a592540338192 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:53 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:36606
I20260923 15:06:53.491993 15587 tablet_replica.cc:330] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: stopping tablet replica
I20260923 15:06:53.492090 15587 raft_consensus.cc:2229] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:53.492200 15587 raft_consensus.cc:2258] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:53.492671 15587 ts_tablet_manager.cc:1905] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:53.492878 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14250
I20260923 15:06:53.494779 15587 ts_tablet_manager.cc:1918] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:53.494858 15587 log.cc:1196] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/482e2bab9f5540ad849a592540338192
I20260923 15:06:53.495121 15587 ts_tablet_manager.cc:1939] T 482e2bab9f5540ad849a592540338192 P c412d6a53bf84c72a8433a17e118bd55: Deleting consensus metadata
I20260923 15:06:53.497061 15588 tablet_replica.cc:330] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: stopping tablet replica
I20260923 15:06:53.497182 15588 raft_consensus.cc:2229] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:53.497303 15588 raft_consensus.cc:2258] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:53.497463 15589 tablet_replica.cc:330] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: stopping tablet replica
I20260923 15:06:53.497557 15589 raft_consensus.cc:2229] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:53.497687 15589 raft_consensus.cc:2258] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:53.498164 15589 ts_tablet_manager.cc:1905] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:06:53.499706 15146 connection.cc:535] server connection from 127.0.0.1:50884 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:06:53.500713 15589 ts_tablet_manager.cc:1918] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:53.500787 15589 log.cc:1196] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/482e2bab9f5540ad849a592540338192
I20260923 15:06:53.501096 15589 ts_tablet_manager.cc:1939] T 482e2bab9f5540ad849a592540338192 P cf6d0e8ef4a746599484cb9dc4f217cd: Deleting consensus metadata
W20260923 15:06:53.501462 15423 connection.cc:441] server connection from 127.0.0.1:36606 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:36606 (request call id 1) could send its response
I20260923 15:06:53.501668 15588 ts_tablet_manager.cc:1905] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:53.502807 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.492224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.7.250.124:40281)
0923 15:06:53.492272 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.492274 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.492378 (+ 104us) server_negotiation.cc:407] Connection header received
0923 15:06:53.492599 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.492600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.492610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.492637 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.494977 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.495783 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.496989 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.497307 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.497380 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.499089 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.499090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.499091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.499098 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.501468 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.501908 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.501910 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.501910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.501975 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.502194 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.502194 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.502195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.502361 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.502467 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.502611 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.502694 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:06:53.503335 15588 ts_tablet_manager.cc:1918] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:53.503407 15588 log.cc:1196] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/482e2bab9f5540ad849a592540338192
I20260923 15:06:53.503631 15588 ts_tablet_manager.cc:1939] T 482e2bab9f5540ad849a592540338192 P e72bdea6a7684fde9af7a8c65f28ed01: Deleting consensus metadata
W20260923 15:06:53.504094 15283 connection.cc:441] server connection from 127.0.0.1:49044 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49044 (request call id 1) could send its response
I20260923 15:06:53.511722 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.504198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.7.250.125:33905)
0923 15:06:53.504251 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.504253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.504498 (+ 245us) server_negotiation.cc:407] Connection header received
0923 15:06:53.504594 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.504595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.504606 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.504634 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.505320 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.506056 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.506989 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.507236 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.507306 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.508697 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.508698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.508699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.508707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.510490 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.510841 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.510842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.510843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.510904 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.511104 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.511105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.511105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.511254 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.511369 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.511524 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.511606 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:06:53.512363 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:53.503966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.7.250.124:40281)
0923 15:06:53.504007 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:53.504009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:53.504115 (+ 106us) server_negotiation.cc:407] Connection header received
0923 15:06:53.504273 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:53.504274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:53.504285 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:53.504315 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:53.505280 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.506039 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.507192 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:53.507447 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:53.507795 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:53.509305 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:53.509306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:53.509307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:53.509313 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:53.511101 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.511452 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.511453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.511453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.511507 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:53.511791 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:53.511792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:53.511792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:53.511928 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:53.512019 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:53.512198 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 15:06:53.512272 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:54.488605 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.488269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:51675)
0923 15:06:54.488483 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.488556 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":53,"threads_started":1}
I20260923 15:06:54.488422 15599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.488007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:56055)
0923 15:06:54.488279 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.488362 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":64,"threads_started":1}
I20260923 15:06:54.488793 15601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.488402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:50395)
0923 15:06:54.488684 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.488753 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":52,"threads_started":1}
W20260923 15:06:54.488905 15535 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:38959 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
W20260923 15:06:54.489018 15260 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:38959 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
W20260923 15:06:54.489043 15398 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:38959 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
I20260923 15:06:54.991917 15602 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader af12aef1b44a4ed8beecad3a7b06b6b1)
I20260923 15:06:54.992084 15602 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } }
I20260923 15:06:54.992414 15602 leader_election.cc:288] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959), b23945c5404141028d4bca48daaba76b (127.7.250.124:40281)
I20260923 15:06:54.992844 15606 raft_consensus.cc:482] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader af12aef1b44a4ed8beecad3a7b06b6b1)
I20260923 15:06:54.992892 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.992670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45866)
0923 15:06:54.992783 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.992842 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":46,"threads_started":1}
W20260923 15:06:54.993037 14874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
I20260923 15:06:54.993121 15606 raft_consensus.cc:504] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } }
I20260923 15:06:54.993503 15606 leader_election.cc:288] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959), f0834107645142c39194500863ee9642 (127.7.250.125:33905)
I20260923 15:06:54.993821 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.993571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45870)
0923 15:06:54.993691 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.993776 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
W20260923 15:06:54.994024 15062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
I20260923 15:06:54.994073 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.993896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45882)
0923 15:06:54.993960 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.993992 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260923 15:06:54.994149 14874 leader_election.cc:334] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959): Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
I20260923 15:06:54.994493 15609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.994340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45890)
0923 15:06:54.994372 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.994399 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 15:06:54.994567 15062 leader_election.cc:334] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959): Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
I20260923 15:06:54.996398 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.992675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:40281 (local address 127.0.0.1:46796)
0923 15:06:54.992790 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.992801 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:54.992860 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:54.993120 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:54.993121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:54.993124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:54.993188 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:54.993191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.994307 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.994307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:54.995495 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:54.995498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.996216 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.996217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:54.996269 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:54.996272 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:54.996316 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 15:06:54.996351 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
I20260923 15:06:54.996443 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.992863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46796 (local address 127.7.250.124:40281)
0923 15:06:54.992954 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:54.992957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:54.992966 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:54.992994 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:54.992994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:54.993006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:54.993086 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:54.993342 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.994155 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.995569 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.996114 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.996210 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:54.996222 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:54.996335 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 15:06:54.996371 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71,"thread_start_us":43,"threads_started":1}
I20260923 15:06:54.996814 15088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0834107645142c39194500863ee9642" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b23945c5404141028d4bca48daaba76b" is_pre_election: true
I20260923 15:06:54.996907 15088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0834107645142c39194500863ee9642 in term 1.
I20260923 15:06:54.997144 14875 leader_election.cc:302] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b23945c5404141028d4bca48daaba76b, f0834107645142c39194500863ee9642; no voters: af12aef1b44a4ed8beecad3a7b06b6b1
I20260923 15:06:54.997298 15602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 15:06:54.997349 15602 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Starting leader election (detected failure of leader af12aef1b44a4ed8beecad3a7b06b6b1)
I20260923 15:06:54.997360 15610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.993622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:33905 (local address 127.0.0.1:60900)
0923 15:06:54.993767 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.993777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:54.993825 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:54.994284 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:54.994285 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:54.994288 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:54.994331 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:54.994332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.995192 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.995193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:54.996397 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:54.996399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.997100 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.997100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:54.997167 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:54.997174 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 15:06:54.997252 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:06:54.997293 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":66,"threads_started":1}
I20260923 15:06:54.997378 15602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:06:54.997428 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.993744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60900 (local address 127.7.250.125:33905)
0923 15:06:54.993988 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:54.993990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:54.993997 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:54.994016 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:54.994017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:54.994026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:54.994083 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:54.994403 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.995124 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.996591 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:54.997027 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:54.997097 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:54.997106 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:54.997258 (+ 152us) server_negotiation.cc:299] Negotiation successful
0923 15:06:54.997301 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":213,"thread_start_us":77,"threads_started":1}
I20260923 15:06:54.997714 14900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b23945c5404141028d4bca48daaba76b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f0834107645142c39194500863ee9642" is_pre_election: true
I20260923 15:06:54.998126 15602 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } }
I20260923 15:06:54.998301 15602 leader_election.cc:288] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [CANDIDATE]: Term 2 election: Requested vote from peers af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959), b23945c5404141028d4bca48daaba76b (127.7.250.124:40281)
I20260923 15:06:54.998309 14900 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b23945c5404141028d4bca48daaba76b in current term 2: Already voted for candidate f0834107645142c39194500863ee9642 in this term.
I20260923 15:06:54.998453 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.998366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45908)
0923 15:06:54.998402 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.998426 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 15:06:54.998484 15088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f0834107645142c39194500863ee9642" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b23945c5404141028d4bca48daaba76b"
I20260923 15:06:54.998574 15088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:06:54.998498 15064 leader_election.cc:302] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b23945c5404141028d4bca48daaba76b; no voters: af12aef1b44a4ed8beecad3a7b06b6b1, f0834107645142c39194500863ee9642
I20260923 15:06:54.998706 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:54.998635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45912)
0923 15:06:54.998669 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:54.998688 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260923 15:06:54.998788 14874 leader_election.cc:334] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959): Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
I20260923 15:06:54.999243 15088 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0834107645142c39194500863ee9642 in term 2.
I20260923 15:06:54.999369 15606 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:06:54.999454 14875 leader_election.cc:302] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b23945c5404141028d4bca48daaba76b, f0834107645142c39194500863ee9642; no voters: af12aef1b44a4ed8beecad3a7b06b6b1
I20260923 15:06:54.999554 15602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:06:54.999738 15602 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [term 2 LEADER]: Becoming Leader. State: Replica: f0834107645142c39194500863ee9642, State: Running, Role: LEADER
I20260923 15:06:54.999841 15602 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } }
I20260923 15:06:55.000193 15613 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0834107645142c39194500863ee9642. Latest consensus state: current_term: 2 leader_uuid: "f0834107645142c39194500863ee9642" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:55.000286 15613 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:55.000465 15615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:55.000864 15615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:55.000922 15615 catalog_manager.cc:1259] Loaded cluster ID: c0ae394f1adb4c1b9dd72b947907434a
I20260923 15:06:55.000973 15615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:55.001135 15615 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:55.001219 15615 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Loaded TSK: 0
I20260923 15:06:55.001369 15615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:55.052557 14890 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:55.054340 14890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:55.055430 15088 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [term 2 FOLLOWER]: Refusing update from remote peer f0834107645142c39194500863ee9642: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260923 15:06:55.055462 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.055290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45922)
0923 15:06:55.055381 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.055423 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260923 15:06:55.055788 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.055699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.0.0.1:45930)
0923 15:06:55.055731 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.055761 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
I20260923 15:06:55.055871 15613 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260923 15:06:55.056001 14874 consensus_peers.cc:487] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 -> Peer af12aef1b44a4ed8beecad3a7b06b6b1 (127.7.250.126:38959): Couldn't send request to peer af12aef1b44a4ed8beecad3a7b06b6b1. Status: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 15:06:55.056564 15606 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0834107645142c39194500863ee9642. Latest consensus state: current_term: 2 leader_uuid: "f0834107645142c39194500863ee9642" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:55.056675 15606 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:55.057147 15602 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f0834107645142c39194500863ee9642" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:55.057257 15602 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:55.057408 15606 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f0834107645142c39194500863ee9642" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "af12aef1b44a4ed8beecad3a7b06b6b1" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38959 } } peers { permanent_uuid: "f0834107645142c39194500863ee9642" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 33905 } attrs { promote: false } } peers { permanent_uuid: "b23945c5404141028d4bca48daaba76b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 40281 } attrs { promote: false } } }
I20260923 15:06:55.057508 15606 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23945c5404141028d4bca48daaba76b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:55.062476 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.059171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42125 (local address 127.0.0.1:49144)
0923 15:06:55.059325 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.059337 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:55.059379 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:55.059661 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:55.059661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:55.059663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:55.059713 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.059715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.060465 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.060466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.061462 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.061464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.062219 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.062300 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.062306 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:06:55.062359 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 15:06:55.062401 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260923 15:06:55.062775 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.059390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.7.250.67:42125)
0923 15:06:55.059533 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:55.059535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:55.059543 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:55.059574 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:55.059574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:55.059583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:55.059659 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:55.059792 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.060390 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.061555 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.062127 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062227 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.062242 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:55.062646 (+ 404us) server_negotiation.cc:299] Negotiation successful
0923 15:06:55.062697 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260923 15:06:55.063181 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.059132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43863 (local address 127.0.0.1:58352)
0923 15:06:55.059210 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.059228 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:55.059282 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:55.059796 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:55.059798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:55.059800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:55.059849 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.059851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.060853 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.060854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.061985 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.061987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.062866 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.062950 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.062954 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:55.062985 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 15:06:55.063033 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":22}
I20260923 15:06:55.063172 15468 tablet_service.cc:1467] Processing CreateTablet for tablet 629cd20ccc7b4d148959f73ab3c69dac (DEFAULT_TABLE table=test-table [id=bfe090ff31d94bae9e003e7f92402f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:55.063318 15468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 629cd20ccc7b4d148959f73ab3c69dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:55.063323 15622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.059243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58352 (local address 127.7.250.65:43863)
0923 15:06:55.059634 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:55.059637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:55.059645 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:55.059672 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:55.059673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:55.059685 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:55.059783 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:55.059937 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.060756 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062082 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.062761 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.063129 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.063147 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:55.063168 (+ 21us) server_negotiation.cc:299] Negotiation successful
0923 15:06:55.063217 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":53,"threads_started":1}
I20260923 15:06:55.063184 15621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.059196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:35141 (local address 127.0.0.1:39594)
0923 15:06:55.059467 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.059474 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:55.059534 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:55.060007 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:55.060008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:55.060010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:55.060044 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.060046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.061013 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.061015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.062094 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.062097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.062870 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.062940 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.062944 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:55.063017 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:06:55.063065 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":4,"thread_start_us":91,"threads_started":1}
I20260923 15:06:55.063758 15193 tablet_service.cc:1467] Processing CreateTablet for tablet 629cd20ccc7b4d148959f73ab3c69dac (DEFAULT_TABLE table=test-table [id=bfe090ff31d94bae9e003e7f92402f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:55.063895 15193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 629cd20ccc7b4d148959f73ab3c69dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:55.064826 15625 tablet_bootstrap.cc:492] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Bootstrap starting.
I20260923 15:06:55.065023 15623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.059260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39594 (local address 127.7.250.66:35141)
0923 15:06:55.059869 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:55.059871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:55.059880 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:55.059909 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:55.059909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:55.059922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:55.059999 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:55.060104 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.060922 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062165 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.062749 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.062866 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.062881 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:55.064897 (+ 2016us) server_negotiation.cc:299] Negotiation successful
0923 15:06:55.064952 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":55,"threads_started":1}
I20260923 15:06:55.065208 15331 tablet_service.cc:1467] Processing CreateTablet for tablet 629cd20ccc7b4d148959f73ab3c69dac (DEFAULT_TABLE table=test-table [id=bfe090ff31d94bae9e003e7f92402f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:55.065359 15331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 629cd20ccc7b4d148959f73ab3c69dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:55.065325 15625 tablet_bootstrap.cc:654] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:55.066062 15625 tablet_bootstrap.cc:492] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: No bootstrap required, opened a new log
I20260923 15:06:55.066134 15625 ts_tablet_manager.cc:1397] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:55.066395 15626 tablet_bootstrap.cc:492] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55: Bootstrap starting.
I20260923 15:06:55.066354 15625 raft_consensus.cc:348] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.066448 15625 raft_consensus.cc:374] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:55.066460 15628 tablet_bootstrap.cc:492] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Bootstrap starting.
I20260923 15:06:55.066481 15625 raft_consensus.cc:729] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf6d0e8ef4a746599484cb9dc4f217cd, State: Initialized, Role: FOLLOWER
I20260923 15:06:55.066606 15625 consensus_queue.cc:260] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.066776 15625 ts_tablet_manager.cc:1428] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:55.066800 15628 tablet_bootstrap.cc:654] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:55.067011 15626 tablet_bootstrap.cc:654] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:55.067239 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.067037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:50355)
0923 15:06:55.067149 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.067187 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":75,"threads_started":1}
I20260923 15:06:55.067680 15536 heartbeater.cc:503] Master 127.7.250.125:33905 was elected leader, sending a full tablet report...
I20260923 15:06:55.067834 15626 tablet_bootstrap.cc:492] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55: No bootstrap required, opened a new log
I20260923 15:06:55.067899 15626 ts_tablet_manager.cc:1397] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:55.068025 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.067733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:44229)
0923 15:06:55.067835 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.067874 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 15:06:55.068084 15626 raft_consensus.cc:348] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.068171 15626 raft_consensus.cc:374] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:55.068209 15626 raft_consensus.cc:729] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c412d6a53bf84c72a8433a17e118bd55, State: Initialized, Role: FOLLOWER
I20260923 15:06:55.068275 15626 consensus_queue.cc:260] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.068396 15628 tablet_bootstrap.cc:492] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: No bootstrap required, opened a new log
I20260923 15:06:55.068456 15628 ts_tablet_manager.cc:1397] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:55.068655 15628 raft_consensus.cc:348] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.068835 15628 raft_consensus.cc:374] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:55.068868 15628 raft_consensus.cc:729] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72bdea6a7684fde9af7a8c65f28ed01, State: Initialized, Role: FOLLOWER
I20260923 15:06:55.068930 15628 consensus_queue.cc:260] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.068979 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.068647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:49295)
0923 15:06:55.068861 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.068907 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":106,"threads_started":1}
I20260923 15:06:55.069082 15628 ts_tablet_manager.cc:1428] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:55.068486 15626 ts_tablet_manager.cc:1428] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:55.069348 15261 heartbeater.cc:503] Master 127.7.250.125:33905 was elected leader, sending a full tablet report...
I20260923 15:06:55.069789 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.069375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:46159)
0923 15:06:55.069539 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.069742 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":73,"threads_started":1}
I20260923 15:06:55.069918 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.069392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:45775)
0923 15:06:55.069840 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.069872 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430}
I20260923 15:06:55.070119 15399 heartbeater.cc:503] Master 127.7.250.125:33905 was elected leader, sending a full tablet report...
I20260923 15:06:55.070425 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.070309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:47749)
0923 15:06:55.070372 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.070397 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260923 15:06:55.106448 15633 raft_consensus.cc:482] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:55.106572 15633 raft_consensus.cc:504] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.107261 15633 leader_election.cc:288] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125), e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:55.110098 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.107426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:60403 (local address 127.7.250.67:42125)
0923 15:06:55.107477 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:55.107479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:55.107566 (+ 87us) server_negotiation.cc:407] Connection header received
0923 15:06:55.107636 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:55.107637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:55.107645 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:55.107706 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:55.107867 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.108329 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.109390 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.109739 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.109823 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.109834 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:55.109994 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 15:06:55.110032 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:06:55.110108 15637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.107214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:42125 (local address 127.7.250.65:60403)
0923 15:06:55.107494 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.107505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:55.107582 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:55.107711 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:55.107711 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:55.107713 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:55.107797 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.107798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.108438 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.108439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.109264 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.109266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.109822 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.109823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.109893 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.109897 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:55.109982 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:06:55.110022 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":51,"thread_start_us":103,"threads_started":1}
I20260923 15:06:55.110163 15623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.107307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:55171 (local address 127.7.250.66:35141)
0923 15:06:55.107355 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:55.107357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:55.107426 (+ 69us) server_negotiation.cc:407] Connection header received
0923 15:06:55.107530 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:55.107531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:55.107543 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:55.107617 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:55.107797 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.108310 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.109344 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.109757 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.109826 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.109837 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:55.110050 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 15:06:55.110086 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:55.110191 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.107130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:35141 (local address 127.7.250.65:55171)
0923 15:06:55.107356 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.107368 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:55.107454 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:55.107619 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:55.107620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:55.107622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:55.107713 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.107722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.108392 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.108393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.109253 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:55.109256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.109891 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.109892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:55.109971 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.109973 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:55.110044 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 15:06:55.110085 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":28}
I20260923 15:06:55.110502 15351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629cd20ccc7b4d148959f73ab3c69dac" candidate_uuid: "c412d6a53bf84c72a8433a17e118bd55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" is_pre_election: true
I20260923 15:06:55.110633 15351 raft_consensus.cc:2454] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c412d6a53bf84c72a8433a17e118bd55 in term 0.
I20260923 15:06:55.110592 15488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629cd20ccc7b4d148959f73ab3c69dac" candidate_uuid: "c412d6a53bf84c72a8433a17e118bd55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" is_pre_election: true
I20260923 15:06:55.110730 15488 raft_consensus.cc:2454] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c412d6a53bf84c72a8433a17e118bd55 in term 0.
I20260923 15:06:55.110816 15145 leader_election.cc:302] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c412d6a53bf84c72a8433a17e118bd55, e72bdea6a7684fde9af7a8c65f28ed01; no voters:
I20260923 15:06:55.110982 15633 raft_consensus.cc:2781] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:55.111044 15633 raft_consensus.cc:482] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:55.111071 15633 raft_consensus.cc:3037] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:55.111783 15633 raft_consensus.cc:504] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.111930 15633 leader_election.cc:288] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [CANDIDATE]: Term 1 election: Requested vote from peers cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125), e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141)
I20260923 15:06:55.112104 15351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629cd20ccc7b4d148959f73ab3c69dac" candidate_uuid: "c412d6a53bf84c72a8433a17e118bd55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72bdea6a7684fde9af7a8c65f28ed01"
I20260923 15:06:55.112097 15488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "629cd20ccc7b4d148959f73ab3c69dac" candidate_uuid: "c412d6a53bf84c72a8433a17e118bd55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd"
I20260923 15:06:55.112198 15351 raft_consensus.cc:3037] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:55.112196 15488 raft_consensus.cc:3037] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:55.112890 15351 raft_consensus.cc:2454] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c412d6a53bf84c72a8433a17e118bd55 in term 1.
I20260923 15:06:55.113096 15145 leader_election.cc:302] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c412d6a53bf84c72a8433a17e118bd55, e72bdea6a7684fde9af7a8c65f28ed01; no voters:
I20260923 15:06:55.113135 15488 raft_consensus.cc:2454] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c412d6a53bf84c72a8433a17e118bd55 in term 1.
I20260923 15:06:55.113186 15633 raft_consensus.cc:2781] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:55.113354 15633 raft_consensus.cc:686] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [term 1 LEADER]: Becoming Leader. State: Replica: c412d6a53bf84c72a8433a17e118bd55, State: Running, Role: LEADER
I20260923 15:06:55.113433 15633 consensus_queue.cc:237] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } }
I20260923 15:06:55.113698 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.113577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:38873)
0923 15:06:55.113634 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.113664 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:06:55.114221 14890 catalog_manager.cc:5495] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 reported cstate change: term changed from 0 to 1, leader changed from <none> to c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "c412d6a53bf84c72a8433a17e118bd55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c412d6a53bf84c72a8433a17e118bd55" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:55.124366 15622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.121491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.7.250.65:43863)
0923 15:06:55.121556 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:55.121558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:55.121567 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:55.121612 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:55.121612 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:55.121631 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:55.121696 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:55.121884 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.122565 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.123382 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:55.123588 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:55.123652 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:55.123845 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:55.123955 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:55.124021 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:55.124150 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 15:06:55.124175 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:06:55.125914 15351 raft_consensus.cc:1261] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Refusing update from remote peer c412d6a53bf84c72a8433a17e118bd55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:55.125989 15488 raft_consensus.cc:1261] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Refusing update from remote peer c412d6a53bf84c72a8433a17e118bd55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:55.126317 15638 consensus_queue.cc:1035] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72bdea6a7684fde9af7a8c65f28ed01" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 35141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:55.126484 15633 consensus_queue.cc:1035] T 629cd20ccc7b4d148959f73ab3c69dac P c412d6a53bf84c72a8433a17e118bd55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf6d0e8ef4a746599484cb9dc4f217cd" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:55.127117 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.126918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:37617)
0923 15:06:55.127045 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.127085 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260923 15:06:55.127843 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.127714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:36929)
0923 15:06:55.127782 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.127810 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 15:06:55.128484 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.128281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:54285)
0923 15:06:55.128419 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.128452 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260923 15:06:55.128937 15631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.128778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.67:39663)
0923 15:06:55.128880 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.128908 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260923 15:06:55.129328 15636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.129242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.66:48071)
0923 15:06:55.129276 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.129301 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 15:06:55.130605 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.129101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:38959 (local address 127.7.250.65:43843)
0923 15:06:55.130540 (+ 1439us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.130570 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:38959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1418}
I20260923 15:06:55.130718 14888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:55.130815 14888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:55.132362 14888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0834107645142c39194500863ee9642: Sending DeleteTablet for 3 replicas of tablet 629cd20ccc7b4d148959f73ab3c69dac
I20260923 15:06:55.132723 15331 tablet_service.cc:1514] Processing DeleteTablet for tablet 629cd20ccc7b4d148959f73ab3c69dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:55 UTC) from {username='oryx', principal='oryx/127.7.250.125@KRBTEST.COM'} at 127.0.0.1:39594
I20260923 15:06:55.132679 15468 tablet_service.cc:1514] Processing DeleteTablet for tablet 629cd20ccc7b4d148959f73ab3c69dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:55 UTC) from {username='oryx', principal='oryx/127.7.250.125@KRBTEST.COM'} at 127.0.0.1:49144
I20260923 15:06:55.133049 15651 tablet_replica.cc:330] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: stopping tablet replica
I20260923 15:06:55.133108 15651 raft_consensus.cc:2229] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:55.133141 15651 raft_consensus.cc:2258] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:55.133256 15650 tablet_replica.cc:330] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: stopping tablet replica
I20260923 15:06:55.133307 15650 raft_consensus.cc:2229] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:55.133352 15650 raft_consensus.cc:2258] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:55.133342 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 15038
I20260923 15:06:55.133639 15650 ts_tablet_manager.cc:1905] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:55.134495 15651 ts_tablet_manager.cc:1905] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:55.135836 15650 ts_tablet_manager.cc:1918] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:55.135921 15650 log.cc:1196] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/629cd20ccc7b4d148959f73ab3c69dac
I20260923 15:06:55.136155 15650 ts_tablet_manager.cc:1939] T 629cd20ccc7b4d148959f73ab3c69dac P cf6d0e8ef4a746599484cb9dc4f217cd: Deleting consensus metadata
I20260923 15:06:55.136520 15651 ts_tablet_manager.cc:1918] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:55.136587 15651 log.cc:1196] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/629cd20ccc7b4d148959f73ab3c69dac
I20260923 15:06:55.136842 15651 ts_tablet_manager.cc:1939] T 629cd20ccc7b4d148959f73ab3c69dac P e72bdea6a7684fde9af7a8c65f28ed01: Deleting consensus metadata
I20260923 15:06:55.136850 14875 catalog_manager.cc:4841] TS cf6d0e8ef4a746599484cb9dc4f217cd (127.7.250.67:42125): tablet 629cd20ccc7b4d148959f73ab3c69dac (table test-table [id=bfe090ff31d94bae9e003e7f92402f58]) successfully deleted
I20260923 15:06:55.137662 14874 catalog_manager.cc:4841] TS e72bdea6a7684fde9af7a8c65f28ed01 (127.7.250.66:35141): tablet 629cd20ccc7b4d148959f73ab3c69dac (table test-table [id=bfe090ff31d94bae9e003e7f92402f58]) successfully deleted
W20260923 15:06:55.139991 14877 connection.cc:535] client connection to 127.7.250.65:43863 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:06:55.140398 15621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:55.140275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43863 (local address 127.0.0.1:58376)
0923 15:06:55.140336 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:55.140371 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:06:55.140401 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 15267
W20260923 15:06:55.140547 14877 catalog_manager.cc:4584] TS c412d6a53bf84c72a8433a17e118bd55 (127.7.250.65:43863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 629cd20ccc7b4d148959f73ab3c69dac: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43863: connect: Connection refused (error 111)
I20260923 15:06:55.146021 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 15404
I20260923 15:06:55.151777 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14749
I20260923 15:06:55.156808 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 14944
2026-09-23T15:06:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4890 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:55 dist-test-slave-bb3x krb5kdc[15654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:55 dist-test-slave-bb3x krb5kdc[15654](info): set up 2 sockets
Sep 23 15:06:55 dist-test-slave-bb3x krb5kdc[15654](info): commencing operation
krb5kdc: starting...
W20260923 15:06:56.208489 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab.
Entry for principal oryx/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab.
I20260923 15:06:56.330947 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39149
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab
--principal=oryx/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:56.420998 15667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:56.421257 15667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:56.421299 15667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:56.422667 15667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:56.422739 15667 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 15:06:56.422765 15667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:56.422806 15667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:56.422842 15667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:56.422873 15667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:56.424537 15667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.126.keytab
--principal=oryx/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:56.424867 15667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:56.425117 15667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:56.430634 15673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.430646 15672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.430645 15675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:56.430955 15667 server_base.cc:1034] running on GCE node
I20260923 15:06:56.431223 15667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:56.431547 15667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:56.432734 15667 hybrid_clock.cc:648] HybridClock initialized: now 1790176016432724 us; error 40 us; skew 500 ppm
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:56.435420 15667 init.cc:377] Logged in from keytab as oryx/127.7.250.126@KRBTEST.COM (short username oryx)
I20260923 15:06:56.435940 15667 webserver.cc:460] Webserver started at http://127.7.250.126:36429/ using document root <none> and password file <none>
I20260923 15:06:56.436188 15667 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:56.436260 15667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:56.436416 15667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:56.437556 15667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "d0ee19e717d9d88369c70c382f9c78b4"
server_key_iv: "fc72b1bb0e797260450ace430d3c55e4"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.437944 15667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "d0ee19e717d9d88369c70c382f9c78b4"
server_key_iv: "fc72b1bb0e797260450ace430d3c55e4"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.439450 15667 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:56.440279 15682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:56.440505 15667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:56.440626 15667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "d0ee19e717d9d88369c70c382f9c78b4"
server_key_iv: "fc72b1bb0e797260450ace430d3c55e4"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.440724 15667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:56.454876 15667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:56.457129 15667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:56.457306 15667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:56.462039 15667 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:39149
I20260923 15:06:56.462121 15745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:39149 every 8 connection(s)
I20260923 15:06:56.462481 15667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:56.463169 15746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:56.465437 15746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Bootstrap starting.
I20260923 15:06:56.466053 15746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:56.466180 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 15667
I20260923 15:06:56.466321 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:06:56.466470 15746 log.cc:824] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:56.466540 8169 external_mini_cluster.cc:1428] Setting key fac433cd3df3f2a943ed261205b6529e
I20260923 15:06:56.467298 15746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: No bootstrap required, opened a new log
I20260923 15:06:56.469909 15746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39149 } }
I20260923 15:06:56.470096 15746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:56.470135 15746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 829ea8f74a6a4d6b86be2ea0b382e1c9, State: Initialized, Role: FOLLOWER
I20260923 15:06:56.470295 15746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39149 } }
I20260923 15:06:56.470383 15746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:56.470412 15746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:56.470455 15746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:56.471068 15746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39149 } }
I20260923 15:06:56.471205 15746 leader_election.cc:302] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 829ea8f74a6a4d6b86be2ea0b382e1c9; no voters:
I20260923 15:06:56.471390 15746 leader_election.cc:288] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:56.471429 15751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:56.471637 15751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [term 1 LEADER]: Becoming Leader. State: Replica: 829ea8f74a6a4d6b86be2ea0b382e1c9, State: Running, Role: LEADER
I20260923 15:06:56.471730 15746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:56.471776 15751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39149 } }
I20260923 15:06:56.472340 15751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 829ea8f74a6a4d6b86be2ea0b382e1c9. Latest consensus state: current_term: 1 leader_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39149 } } }
I20260923 15:06:56.472605 15751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:56.472530 15753 sys_catalog.cc:455] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "829ea8f74a6a4d6b86be2ea0b382e1c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39149 } } }
I20260923 15:06:56.473074 15753 sys_catalog.cc:458] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:56.473214 15760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:56.473650 15760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:56.476037 15760 catalog_manager.cc:1347] Generated new cluster ID: 8a6a82b1b0c14785b3c38c700d8c8466
I20260923 15:06:56.476091 15760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:56.478295 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.467055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.7.250.126:39149)
0923 15:06:56.467265 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:56.467273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:56.467299 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:06:56.467783 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:56.467791 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:56.467837 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:56.467947 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:56.468775 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.469671 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.470796 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.471276 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.471415 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.474435 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:56.474439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:56.474440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:56.474452 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:56.476618 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.477027 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.477030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.477030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.477142 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.477364 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.477364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.477365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.477653 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:56.477807 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:56.477928 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 15:06:56.478067 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260923 15:06:56.487402 15760 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:56.488121 15760 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:56.496191 15760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Generated new TSK 0
I20260923 15:06:56.496403 15760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab.
Entry for principal oryx/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab.
I20260923 15:06:56.529707 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39149
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:56.615245 15772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:56.615460 15772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:56.615484 15772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:56.616771 15772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:56.616828 15772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:56.616850 15772 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:56.616894 15772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:06:56.618525 15772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:39149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:56.618775 15772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:56.619037 15772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:56.624353 15777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.624363 15780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.624363 15778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:56.624588 15772 server_base.cc:1034] running on GCE node
I20260923 15:06:56.624905 15772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:56.625169 15772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:56.626382 15772 hybrid_clock.cc:648] HybridClock initialized: now 1790176016626374 us; error 54 us; skew 500 ppm
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:56.634917 15772 init.cc:377] Logged in from keytab as oryx/127.7.250.65@KRBTEST.COM (short username oryx)
I20260923 15:06:56.635466 15772 webserver.cc:460] Webserver started at http://127.7.250.65:34143/ using document root <none> and password file <none>
I20260923 15:06:56.635710 15772 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:56.635782 15772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:56.635937 15772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:56.637068 15772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "c716b31974d9b0dc19f803151e382e7d"
server_key_iv: "8d37748e5d8e1ccfbbd4fb4210d4590f"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.637449 15772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "c716b31974d9b0dc19f803151e382e7d"
server_key_iv: "8d37748e5d8e1ccfbbd4fb4210d4590f"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.638942 15772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:56.639819 15787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:56.640130 15772 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:56.640271 15772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "c716b31974d9b0dc19f803151e382e7d"
server_key_iv: "8d37748e5d8e1ccfbbd4fb4210d4590f"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.640365 15772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:56.654309 15772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:56.656479 15772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:56.656656 15772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:56.657272 15772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:56.657325 15772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:56.657357 15772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:56.657379 15772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:56.664718 15772 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:34457
I20260923 15:06:56.664784 15902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:34457 every 8 connection(s)
I20260923 15:06:56.665141 15772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:06:56.665339 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 15772
I20260923 15:06:56.665462 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:06:56.665693 8169 external_mini_cluster.cc:1428] Setting key ed3c99335ef39af633d2293f34120457
I20260923 15:06:56.668252 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.657706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:48729 (local address 127.7.250.126:39149)
0923 15:06:56.657762 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:56.657765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:56.657971 (+ 206us) server_negotiation.cc:407] Connection header received
0923 15:06:56.658686 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:56.658688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:56.658702 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:56.658749 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:56.659505 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.660069 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.661282 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.661511 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.661601 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.664699 (+ 3098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:56.664701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:56.664701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:56.664718 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:56.666565 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.667013 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.667014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.667015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.667081 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.667465 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.667466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.667467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.667622 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:56.667704 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:56.668110 (+ 406us) server_negotiation.cc:299] Negotiation successful
0923 15:06:56.668188 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 15:06:56.668637 15796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.657665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.7.250.65:48729)
0923 15:06:56.657829 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:56.657870 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:56.658532 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:56.658840 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:56.658844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:56.659171 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:56.659394 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.659399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.660201 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.660202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.661175 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.661178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.661626 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.661627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.661802 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.662502 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:56.662513 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:56.664580 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:56.666666 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.666673 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.666674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.666921 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.667275 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.667276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.667276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.667385 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.667755 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:56.667763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:56.667951 (+ 188us) client_negotiation.cc:769] Sending connection context
0923 15:06:56.668085 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 15:06:56.668368 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260923 15:06:56.669178 15903 heartbeater.cc:347] Connected to a master server at 127.7.250.126:39149
I20260923 15:06:56.669284 15903 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:56.669485 15903 heartbeater.cc:511] Master 127.7.250.126:39149 requested a full tablet report, sending...
I20260923 15:06:56.670055 15698 ts_manager.cc:194] Registered new tserver with Master: 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457)
I20260923 15:06:56.671072 15698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:48729
WARNING: no policy specified for oryx/127.7.250.66@KRBTEST.COM; defaulting to no policy
I20260923 15:06:56.676596 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.670487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:45739 (local address 127.7.250.126:39149)
0923 15:06:56.670531 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:56.670534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:56.670716 (+ 182us) server_negotiation.cc:407] Connection header received
0923 15:06:56.670788 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:56.670788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:56.670798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:56.670846 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:56.671203 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.671838 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.672770 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.672966 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.673048 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.674023 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:56.674025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:56.674025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:56.674033 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:56.675446 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.675810 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.675811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.675812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.675867 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.676089 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.676090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.676090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.676202 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:56.676278 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:56.676429 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 15:06:56.676501 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:56.676608 15910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.670495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.7.250.65:45739)
0923 15:06:56.670624 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:56.670633 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:56.670730 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:56.670847 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:56.670848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:56.671035 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:56.671085 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.671087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.671933 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.671934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.672669 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.672671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.673051 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.673052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.673115 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.673626 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:56.673630 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:56.673937 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:56.675547 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.675548 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.675549 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.675720 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.675963 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.675964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.675964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.676024 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.676269 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:56.676270 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:56.676314 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 15:06:56.676403 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 15:06:56.676504 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab.
Entry for principal oryx/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab.
I20260923 15:06:56.697860 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39149
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:56.779753 15912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:56.779953 15912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:56.779976 15912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:56.781041 15912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:56.781075 15912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:56.781095 15912 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:56.781112 15912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:06:56.782559 15912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:39149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:56.782742 15912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:56.782920 15912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:56.787982 15920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:56.788141 15912 server_base.cc:1034] running on GCE node
W20260923 15:06:56.787959 15917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.787954 15918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:56.788597 15912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:56.788902 15912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:56.790091 15912 hybrid_clock.cc:648] HybridClock initialized: now 1790176016790072 us; error 30 us; skew 500 ppm
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:56.792847 15912 init.cc:377] Logged in from keytab as oryx/127.7.250.66@KRBTEST.COM (short username oryx)
I20260923 15:06:56.793355 15912 webserver.cc:460] Webserver started at http://127.7.250.66:34287/ using document root <none> and password file <none>
I20260923 15:06:56.793627 15912 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:56.793713 15912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:56.793869 15912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:56.795089 15912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "b3a7b570fc0c642c276f96d4e8fa74f1"
server_key_iv: "94f34c149534b9c61c8bc19af31bc157"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.795481 15912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "b3a7b570fc0c642c276f96d4e8fa74f1"
server_key_iv: "94f34c149534b9c61c8bc19af31bc157"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.796939 15912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:06:56.797821 15927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:56.798017 15912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:56.798134 15912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "b3a7b570fc0c642c276f96d4e8fa74f1"
server_key_iv: "94f34c149534b9c61c8bc19af31bc157"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.798249 15912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:56.832284 15912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:56.834494 15912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:56.834673 15912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:56.835263 15912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:56.835330 15912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:56.835408 15912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:56.835443 15912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:56.843027 15912 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:46265
I20260923 15:06:56.843072 16043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:46265 every 8 connection(s)
I20260923 15:06:56.843528 15912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:06:56.845881 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.835901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:55289 (local address 127.7.250.126:39149)
0923 15:06:56.835965 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:56.835967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:56.836161 (+ 194us) server_negotiation.cc:407] Connection header received
0923 15:06:56.836766 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:56.836768 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:56.836780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:56.836837 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:56.837782 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.838535 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.839570 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.839776 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.839865 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.842606 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:56.842607 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:56.842608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:56.842616 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:56.844400 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.844865 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.844866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.844866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.844925 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.845210 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.845211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.845211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.845339 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:56.845418 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:56.845693 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 15:06:56.845776 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:06:56.846086 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.835858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.7.250.66:55289)
0923 15:06:56.836032 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:56.836078 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:56.836648 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:56.836912 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:56.836916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:56.837396 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:56.837695 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.837700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.838645 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.838646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.839468 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.839471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.839890 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.839891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.840039 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.840620 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:56.840629 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:56.842495 (+ 1866us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:56.844491 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.844498 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.844501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.844784 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.845001 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.845002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.845003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.845117 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.845407 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:56.845414 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:56.845555 (+ 141us) client_negotiation.cc:769] Sending connection context
0923 15:06:56.845664 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:06:56.845846 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260923 15:06:56.846529 16044 heartbeater.cc:347] Connected to a master server at 127.7.250.126:39149
I20260923 15:06:56.846635 16044 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:56.846823 16044 heartbeater.cc:511] Master 127.7.250.126:39149 requested a full tablet report, sending...
I20260923 15:06:56.847263 15699 ts_manager.cc:194] Registered new tserver with Master: e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265)
I20260923 15:06:56.847664 15699 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:55289
I20260923 15:06:56.852991 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.847724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:59461 (local address 127.7.250.126:39149)
0923 15:06:56.847856 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:56.847858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:56.847866 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:56.847907 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:56.847908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:56.847917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:56.847976 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:56.848313 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.848828 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.849537 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.849719 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.849774 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.850647 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:56.850649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:56.850649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:56.850655 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:56.851854 (+ 1199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.852163 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.852164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.852164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.852267 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:56.852531 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:56.852532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:56.852532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:56.852690 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:56.852758 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:56.852873 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 15:06:56.852932 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260923 15:06:56.853008 16051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:56.847642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.7.250.66:59461)
0923 15:06:56.847780 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:56.847791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:56.847845 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:56.847981 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:56.847982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:56.848162 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:56.848235 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.848236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.848919 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.848920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.849463 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:56.849464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:56.849793 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:56.849794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:56.849845 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:56.850201 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:56.850206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:56.850571 (+ 365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:56.851929 (+ 1358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.851929 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.851930 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.852096 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.852340 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:56.852340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:56.852341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:56.852398 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:56.852761 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:56.852762 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:56.852800 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 15:06:56.852856 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:06:56.852935 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260923 15:06:56.853652 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 15912
I20260923 15:06:56.853785 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:06:56.853995 8169 external_mini_cluster.cc:1428] Setting key 998d9f5ad6264e060d45bcfec2d05edb
WARNING: no policy specified for oryx/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab.
Entry for principal oryx/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab.
I20260923 15:06:56.880998 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:39149
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:35991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:56.967113 16054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:56.967326 16054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:56.967353 16054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:56.968556 16054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:56.968614 16054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:56.968647 16054 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:06:56.968667 16054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:06:56.970366 16054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:35991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.250.126:39149
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:56.970628 16054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:56.970868 16054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:56.976492 16059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.976492 16062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:56.976766 16060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:56.977769 16054 server_base.cc:1034] running on GCE node
I20260923 15:06:56.978027 16054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:56.978339 16054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:56.979548 16054 hybrid_clock.cc:648] HybridClock initialized: now 1790176016979524 us; error 40 us; skew 500 ppm
Sep 23 15:06:56 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:56.982306 16054 init.cc:377] Logged in from keytab as oryx/127.7.250.67@KRBTEST.COM (short username oryx)
I20260923 15:06:56.982813 16054 webserver.cc:460] Webserver started at http://127.7.250.67:44459/ using document root <none> and password file <none>
I20260923 15:06:56.983054 16054 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:56.983103 16054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:56.983246 16054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:56.984381 16054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "e105d1ea5a904c47959ce1f02e76c8c2"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "adf6560612dd2d740552b8b78ebdb583"
server_key_iv: "9f4436d0a718627c5a8145571a1318de"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.984766 16054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "e105d1ea5a904c47959ce1f02e76c8c2"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "adf6560612dd2d740552b8b78ebdb583"
server_key_iv: "9f4436d0a718627c5a8145571a1318de"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.986140 16054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:06:56.987042 16069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:56.987341 16054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:56.987453 16054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "e105d1ea5a904c47959ce1f02e76c8c2"
format_stamp: "Formatted at 2026-09-23 15:06:56 on dist-test-slave-bb3x"
server_key: "adf6560612dd2d740552b8b78ebdb583"
server_key_iv: "9f4436d0a718627c5a8145571a1318de"
server_key_version: "encryptionkey@0"
I20260923 15:06:56.987532 16054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:57.001919 16054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:57.004482 16054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:57.004670 16054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:57.005337 16054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:06:57.005398 16054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.005443 16054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:06:57.005460 16054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:57.013428 16054 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:33891
I20260923 15:06:57.013500 16185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:33891 every 8 connection(s)
I20260923 15:06:57.013880 16054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:06:57.016106 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.005811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:42497 (local address 127.7.250.126:39149)
0923 15:06:57.005895 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.005898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.006208 (+ 310us) server_negotiation.cc:407] Connection header received
0923 15:06:57.007009 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.007011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.007025 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.007087 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.008039 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.008692 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.010294 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.010495 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.010581 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.012831 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:57.012832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:57.012833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:57.012843 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:57.014766 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.015189 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.015190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.015191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.015256 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.015494 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.015495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.015495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.015609 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:57.015671 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.015954 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.016030 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 15:06:57.016207 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16054
I20260923 15:06:57.016288 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.005810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.7.250.67:42497)
0923 15:06:57.006036 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.006103 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.006889 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.007166 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.007171 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.007586 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:57.007900 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.007906 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.008833 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.008834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.010179 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.010181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.010602 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.010603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.010739 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.011280 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:57.011288 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:57.012741 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:57.014843 (+ 2102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:57.014847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:57.014849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:57.015058 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:57.015331 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:57.015332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:57.015332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:57.015430 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:57.015671 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:57.015676 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:57.015797 (+ 121us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.015885 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.016007 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":93,"threads_started":1}
I20260923 15:06:57.016311 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:06:57.016516 8169 external_mini_cluster.cc:1428] Setting key 87dc7c2c38f7075e2f78929da4979fa9
I20260923 15:06:57.016790 16186 heartbeater.cc:347] Connected to a master server at 127.7.250.126:39149
I20260923 15:06:57.016919 16186 heartbeater.cc:464] Registering TS with master...
I20260923 15:06:57.017145 16186 heartbeater.cc:511] Master 127.7.250.126:39149 requested a full tablet report, sending...
I20260923 15:06:57.017539 15699 ts_manager.cc:194] Registered new tserver with Master: e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891)
I20260923 15:06:57.018028 15699 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:42497
I20260923 15:06:57.023729 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.018218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:46093 (local address 127.7.250.126:39149)
0923 15:06:57.018280 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.018282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.018383 (+ 101us) server_negotiation.cc:407] Connection header received
0923 15:06:57.018439 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.018440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.018450 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.018505 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.018834 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.019318 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.020026 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.020220 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.020303 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.021318 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:57.021319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:57.021320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:57.021327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:57.022587 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.022937 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.022938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.022939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.022995 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.023213 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.023214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.023215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.023321 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:57.023400 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.023564 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.023636 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:06:57.023753 16192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.018131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.7.250.67:46093)
0923 15:06:57.018313 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.018324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.018380 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.018520 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.018521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.018706 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:57.018759 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.018760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.019417 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.019418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.019930 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.019931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.020306 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.020307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.020379 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.020846 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:57.020850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:57.021116 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:57.022684 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:57.022685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:57.022686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:57.022852 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:57.023061 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:57.023062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:57.023062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:57.023127 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:57.023405 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:57.023406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:57.023449 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.023534 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.023638 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260923 15:06:57.028213 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:06:57.049422 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:57.043859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36300 (local address 127.7.250.126:39149)
0923 15:06:57.043901 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.043903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.043915 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:57.043947 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.043948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.043959 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.044029 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.044627 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.045273 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.045995 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.046176 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.046241 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.049352 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36300: BlockingRecv error: recv got EOF from 127.0.0.1:36300 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176017, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:57.057554 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.050361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36304 (local address 127.7.250.126:39149)
0923 15:06:57.050424 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.050426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.050433 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:57.050454 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.050455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.050464 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.050520 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.050937 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.051475 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.052136 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.052372 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.052433 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.054963 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:57.054965 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:57.054966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:57.054972 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:57.056417 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.056796 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.056796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.056797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.056846 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.057084 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.057086 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.057086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.057222 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:57.057311 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.057408 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.057472 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:06:57.059420 15699 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:57.060930 15699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:57.067755 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.063688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33262 (local address 127.7.250.67:33891)
0923 15:06:57.063845 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.063851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.063860 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:57.063899 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.063901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.063917 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.063995 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.064221 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.065414 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.066735 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.067238 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.067275 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.067323 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.067584 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.067642 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260923 15:06:57.067750 16208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.063380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33891 (local address 127.0.0.1:33262)
0923 15:06:57.063558 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.063602 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.063680 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.064021 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.064025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.064027 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.064101 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.064103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.065500 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.065501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.066639 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.066641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.067372 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.067373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.067481 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.067497 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.067569 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.067639 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":57,"thread_start_us":59,"threads_started":1}
I20260923 15:06:57.067824 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.063381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46265 (local address 127.0.0.1:51758)
0923 15:06:57.063546 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.063583 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.063861 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.064293 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.064294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.064297 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.064348 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.064350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.065230 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.065231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.066619 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.066622 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.067513 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.067514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.067581 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.067585 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.067617 (+ 32us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.067659 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":50,"thread_start_us":83,"threads_started":1}
I20260923 15:06:57.068044 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.063777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46044 (local address 127.7.250.65:34457)
0923 15:06:57.064110 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.064115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.064124 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:57.064159 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.064161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.064177 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.064382 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.064435 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.065447 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.066635 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.067356 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.067436 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.067511 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.067904 (+ 393us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.067945 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":78,"threads_started":1}
I20260923 15:06:57.068069 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.063756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.7.250.66:46265)
0923 15:06:57.064116 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.064121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.064131 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:57.064166 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.064168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.064184 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.064419 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.064474 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.065122 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.066721 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.067420 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.067796 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.067901 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.067926 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.067965 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":107,"threads_started":1}
I20260923 15:06:57.068831 15978 tablet_service.cc:1467] Processing CreateTablet for tablet 48dd5909b437492c9e9506675117e885 (DEFAULT_TABLE table=test-table [id=617763c660224c8abaff4ed6dd3e3125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.068850 16119 tablet_service.cc:1467] Processing CreateTablet for tablet 48dd5909b437492c9e9506675117e885 (DEFAULT_TABLE table=test-table [id=617763c660224c8abaff4ed6dd3e3125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.069188 15978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48dd5909b437492c9e9506675117e885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.069195 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48dd5909b437492c9e9506675117e885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.067765 16210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.063581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:34457 (local address 127.0.0.1:46044)
0923 15:06:57.063940 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.063948 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.064006 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.064270 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.064271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.064272 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.064314 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.064316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.065548 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.065549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.066542 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.066545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.067446 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.067446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.067495 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.067497 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.067569 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.067632 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":66,"threads_started":1}
I20260923 15:06:57.071025 15837 tablet_service.cc:1467] Processing CreateTablet for tablet 48dd5909b437492c9e9506675117e885 (DEFAULT_TABLE table=test-table [id=617763c660224c8abaff4ed6dd3e3125]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.071382 15837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48dd5909b437492c9e9506675117e885. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.072544 16214 tablet_bootstrap.cc:492] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Bootstrap starting.
I20260923 15:06:57.072970 16215 tablet_bootstrap.cc:492] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Bootstrap starting.
I20260923 15:06:57.073138 16214 tablet_bootstrap.cc:654] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.073400 16214 log.cc:824] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:57.073657 16215 tablet_bootstrap.cc:654] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.074000 16215 log.cc:824] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:57.074080 16214 tablet_bootstrap.cc:492] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: No bootstrap required, opened a new log
I20260923 15:06:57.074124 16214 ts_tablet_manager.cc:1397] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:57.074765 16215 tablet_bootstrap.cc:492] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: No bootstrap required, opened a new log
I20260923 15:06:57.074841 16215 ts_tablet_manager.cc:1397] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:57.075210 16218 tablet_bootstrap.cc:492] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Bootstrap starting.
I20260923 15:06:57.075755 16218 tablet_bootstrap.cc:654] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.076041 16218 log.cc:824] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:57.076391 16214 raft_consensus.cc:348] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.076565 16214 raft_consensus.cc:374] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.076598 16214 raft_consensus.cc:729] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56de0f25a994bd8aa4f57f7afbdd54d, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.076772 16218 tablet_bootstrap.cc:492] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: No bootstrap required, opened a new log
I20260923 15:06:57.076737 16214 consensus_queue.cc:260] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.076853 16218 ts_tablet_manager.cc:1397] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:57.076808 16215 raft_consensus.cc:348] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.076959 16215 raft_consensus.cc:374] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.077000 16215 raft_consensus.cc:729] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.077013 16214 ts_tablet_manager.cc:1428] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:57.077116 16215 consensus_queue.cc:260] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.077147 16044 heartbeater.cc:503] Master 127.7.250.126:39149 was elected leader, sending a full tablet report...
I20260923 15:06:57.077376 16215 ts_tablet_manager.cc:1428] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:06:57.077517 16186 heartbeater.cc:503] Master 127.7.250.126:39149 was elected leader, sending a full tablet report...
I20260923 15:06:57.078922 16218 raft_consensus.cc:348] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.079038 16218 raft_consensus.cc:374] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.079058 16218 raft_consensus.cc:729] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005c1f4c7adf46f5bf1abfd6b4078bcd, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.079164 16218 consensus_queue.cc:260] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.079421 16218 ts_tablet_manager.cc:1428] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:06:57.079448 15903 heartbeater.cc:503] Master 127.7.250.126:39149 was elected leader, sending a full tablet report...
W20260923 15:06:57.094915 16045 tablet.cc:2369] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:06:57.166069 15904 tablet.cc:2369] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:06:57.183874 16221 raft_consensus.cc:482] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:57.183982 16221 raft_consensus.cc:504] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.184278 16221 leader_election.cc:288] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265), 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457)
I20260923 15:06:57.187291 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.184569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:56199 (local address 127.7.250.66:46265)
0923 15:06:57.184613 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.184615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.184624 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:57.184705 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.184705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.184717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.184774 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.184967 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.185534 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.186573 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.186934 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.187012 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.187022 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.187182 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.187221 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:06:57.187300 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.184667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:60271 (local address 127.7.250.65:34457)
0923 15:06:57.184712 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.184714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.184813 (+ 99us) server_negotiation.cc:407] Connection header received
0923 15:06:57.184849 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.184850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.184860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.184927 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.185090 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.185554 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.186575 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.186941 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.187028 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.187038 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.187191 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.187230 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:06:57.187325 16078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.184467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46265 (local address 127.7.250.67:56199)
0923 15:06:57.184531 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.184545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.184609 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.184813 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.184814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.184817 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.184893 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.184895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.185620 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.185621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.186487 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.186489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.187010 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.187011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.187107 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.187109 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.187178 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.187227 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":2}
I20260923 15:06:57.187454 16223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.184589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:34457 (local address 127.7.250.67:60271)
0923 15:06:57.184736 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.184752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.184803 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.184953 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.184954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.184956 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.185023 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.185025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.185675 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.185675 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.186490 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.186492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.187041 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.187042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.187105 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.187108 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.187173 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.187211 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260923 15:06:57.187750 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dd5909b437492c9e9506675117e885" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" is_pre_election: true
I20260923 15:06:57.187749 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dd5909b437492c9e9506675117e885" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" is_pre_election: true
I20260923 15:06:57.187906 15857 raft_consensus.cc:2454] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 0.
I20260923 15:06:57.187918 15998 raft_consensus.cc:2454] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 0.
I20260923 15:06:57.188156 16070 leader_election.cc:302] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e105d1ea5a904c47959ce1f02e76c8c2, e56de0f25a994bd8aa4f57f7afbdd54d; no voters:
I20260923 15:06:57.188287 16221 raft_consensus.cc:2781] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:57.188341 16221 raft_consensus.cc:482] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:57.188357 16221 raft_consensus.cc:3037] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.189278 16221 raft_consensus.cc:504] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.189381 16221 leader_election.cc:288] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 election: Requested vote from peers e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265), 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457)
I20260923 15:06:57.189575 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dd5909b437492c9e9506675117e885" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
I20260923 15:06:57.189622 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dd5909b437492c9e9506675117e885" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
I20260923 15:06:57.189654 15998 raft_consensus.cc:3037] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.189710 15857 raft_consensus.cc:3037] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.190467 15998 raft_consensus.cc:2454] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 1.
I20260923 15:06:57.190587 15857 raft_consensus.cc:2454] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 1.
I20260923 15:06:57.190645 16070 leader_election.cc:302] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e105d1ea5a904c47959ce1f02e76c8c2, e56de0f25a994bd8aa4f57f7afbdd54d; no voters:
I20260923 15:06:57.190723 16221 raft_consensus.cc:2781] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:57.190864 16221 raft_consensus.cc:686] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 LEADER]: Becoming Leader. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Running, Role: LEADER
I20260923 15:06:57.190982 16221 consensus_queue.cc:237] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } }
I20260923 15:06:57.191896 15699 catalog_manager.cc:5495] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:57.217206 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.214709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.7.250.67:33891)
0923 15:06:57.214773 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.214775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.214787 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:57.214830 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.214832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.214850 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.214923 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.215039 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.215603 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.216386 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.216588 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.216646 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.216800 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.216965 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.217035 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.217115 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.217135 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:06:57.219216 15998 raft_consensus.cc:1261] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Refusing update from remote peer e105d1ea5a904c47959ce1f02e76c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.219285 15857 raft_consensus.cc:1261] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Refusing update from remote peer e105d1ea5a904c47959ce1f02e76c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.219564 16224 consensus_queue.cc:1035] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.219728 16221 consensus_queue.cc:1035] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.222441 16228 mvcc.cc:205] Tried to move back new op lower bound from 7332560966527107072 to 7332560966415101952. Current Snapshot: MvccSnapshot[applied={T|T < 7332560966527107072}]
I20260923 15:06:57.222694 16229 mvcc.cc:205] Tried to move back new op lower bound from 7332560966527107072 to 7332560966415101952. Current Snapshot: MvccSnapshot[applied={T|T < 7332560966527107072}]
I20260923 15:06:57.224740 15698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.224843 15698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.225701 15698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Sending DeleteTablet for 3 replicas of tablet 48dd5909b437492c9e9506675117e885
I20260923 15:06:57.226063 16119 tablet_service.cc:1514] Processing DeleteTablet for tablet 48dd5909b437492c9e9506675117e885 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:33262
I20260923 15:06:57.226168 15837 tablet_service.cc:1514] Processing DeleteTablet for tablet 48dd5909b437492c9e9506675117e885 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:46044
I20260923 15:06:57.226244 15698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36304:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:57.226408 15698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:57.226497 15978 tablet_service.cc:1514] Processing DeleteTablet for tablet 48dd5909b437492c9e9506675117e885 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:51758
I20260923 15:06:57.226617 16237 tablet_replica.cc:330] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: stopping tablet replica
I20260923 15:06:57.226882 16237 raft_consensus.cc:2229] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.227005 16237 raft_consensus.cc:2258] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.227464 16237 ts_tablet_manager.cc:1905] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.227871 16236 tablet_replica.cc:330] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: stopping tablet replica
I20260923 15:06:57.228018 16236 raft_consensus.cc:2229] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:57.228224 16236 raft_consensus.cc:2258] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.228262 16119 tablet_service.cc:1467] Processing CreateTablet for tablet 0c60282ff35b48f080fc7dd11642fafa (DEFAULT_TABLE table=test-table [id=18c5d13f1e4f46a6977daff66df6a54e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.228410 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c60282ff35b48f080fc7dd11642fafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.228647 15837 tablet_service.cc:1467] Processing CreateTablet for tablet 0c60282ff35b48f080fc7dd11642fafa (DEFAULT_TABLE table=test-table [id=18c5d13f1e4f46a6977daff66df6a54e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.228775 15837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c60282ff35b48f080fc7dd11642fafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.229215 16238 tablet_replica.cc:330] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: stopping tablet replica
I20260923 15:06:57.229321 16238 raft_consensus.cc:2229] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.229429 16238 raft_consensus.cc:2258] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.229506 15978 tablet_service.cc:1467] Processing CreateTablet for tablet 0c60282ff35b48f080fc7dd11642fafa (DEFAULT_TABLE table=test-table [id=18c5d13f1e4f46a6977daff66df6a54e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.229592 16237 ts_tablet_manager.cc:1918] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:57.229637 15978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c60282ff35b48f080fc7dd11642fafa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.229672 16237 log.cc:1196] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/48dd5909b437492c9e9506675117e885
I20260923 15:06:57.229908 16215 tablet_bootstrap.cc:492] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Bootstrap starting.
I20260923 15:06:57.229944 16237 ts_tablet_manager.cc:1939] T 48dd5909b437492c9e9506675117e885 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting consensus metadata
I20260923 15:06:57.230366 16215 tablet_bootstrap.cc:654] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.230983 16215 tablet_bootstrap.cc:492] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: No bootstrap required, opened a new log
I20260923 15:06:57.231029 16215 ts_tablet_manager.cc:1397] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:57.231097 16214 tablet_bootstrap.cc:492] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Bootstrap starting.
I20260923 15:06:57.231170 16215 raft_consensus.cc:348] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.231240 16215 raft_consensus.cc:374] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.231259 16215 raft_consensus.cc:729] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.231303 16215 consensus_queue.cc:260] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.231401 16215 ts_tablet_manager.cc:1428] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:57.231536 16214 tablet_bootstrap.cc:654] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.231750 16238 ts_tablet_manager.cc:1905] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.232187 15683 catalog_manager.cc:4841] TS 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457): tablet 48dd5909b437492c9e9506675117e885 (table test-table [id=617763c660224c8abaff4ed6dd3e3125]) successfully deleted
I20260923 15:06:57.232969 16236 ts_tablet_manager.cc:1905] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.233187 16218 tablet_bootstrap.cc:492] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Bootstrap starting.
I20260923 15:06:57.233580 16218 tablet_bootstrap.cc:654] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.234156 16236 ts_tablet_manager.cc:1918] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:57.234216 16236 log.cc:1196] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/48dd5909b437492c9e9506675117e885
I20260923 15:06:57.234247 16218 tablet_bootstrap.cc:492] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: No bootstrap required, opened a new log
I20260923 15:06:57.234319 16218 ts_tablet_manager.cc:1397] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:57.234457 16236 ts_tablet_manager.cc:1939] T 48dd5909b437492c9e9506675117e885 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting consensus metadata
I20260923 15:06:57.234699 16218 raft_consensus.cc:348] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.234781 16218 raft_consensus.cc:374] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.234804 16218 raft_consensus.cc:729] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005c1f4c7adf46f5bf1abfd6b4078bcd, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.234848 16218 consensus_queue.cc:260] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.234912 16238 ts_tablet_manager.cc:1918] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:57.234956 16218 ts_tablet_manager.cc:1428] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:57.234992 16238 log.cc:1196] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/48dd5909b437492c9e9506675117e885
I20260923 15:06:57.235191 15684 catalog_manager.cc:4841] TS e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891): tablet 48dd5909b437492c9e9506675117e885 (table test-table [id=617763c660224c8abaff4ed6dd3e3125]) successfully deleted
I20260923 15:06:57.235267 16238 ts_tablet_manager.cc:1939] T 48dd5909b437492c9e9506675117e885 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting consensus metadata
I20260923 15:06:57.235630 15683 catalog_manager.cc:4841] TS e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265): tablet 48dd5909b437492c9e9506675117e885 (table test-table [id=617763c660224c8abaff4ed6dd3e3125]) successfully deleted
I20260923 15:06:57.235939 16214 tablet_bootstrap.cc:492] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: No bootstrap required, opened a new log
I20260923 15:06:57.236001 16214 ts_tablet_manager.cc:1397] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 15:06:57.236148 16214 raft_consensus.cc:348] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.236235 16214 raft_consensus.cc:374] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.236260 16214 raft_consensus.cc:729] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56de0f25a994bd8aa4f57f7afbdd54d, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.236308 16214 consensus_queue.cc:260] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.236397 16214 ts_tablet_manager.cc:1428] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:06:57.246855 16220 raft_consensus.cc:482] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:57.246994 16220 raft_consensus.cc:504] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.247439 16220 leader_election.cc:288] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457), e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891)
I20260923 15:06:57.250571 15936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.247586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:34457 (local address 127.7.250.66:46507)
0923 15:06:57.247709 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.247719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.247771 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.247942 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.247943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.247944 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.248015 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.248017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.248734 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.248735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.249665 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.249668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.250234 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.250235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.250379 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.250382 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.250451 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.250495 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":6}
I20260923 15:06:57.250588 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.247639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:46507 (local address 127.7.250.65:34457)
0923 15:06:57.247690 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.247693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.247753 (+ 60us) server_negotiation.cc:407] Connection header received
0923 15:06:57.247835 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.247836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.247847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.247891 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.248093 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.248591 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.249757 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.250133 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.250234 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.250244 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.250473 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.250530 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:06:57.250833 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c60282ff35b48f080fc7dd11642fafa" candidate_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" is_pre_election: true
I20260923 15:06:57.250928 15857 raft_consensus.cc:2454] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56de0f25a994bd8aa4f57f7afbdd54d in term 0.
I20260923 15:06:57.251228 15928 leader_election.cc:302] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e56de0f25a994bd8aa4f57f7afbdd54d; no voters:
I20260923 15:06:57.251417 16220 raft_consensus.cc:2781] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:57.251464 16220 raft_consensus.cc:482] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:57.251492 16220 raft_consensus.cc:3037] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.251896 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.247918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:56851 (local address 127.7.250.67:33891)
0923 15:06:57.247970 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.247972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.247980 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:57.248006 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.248007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.248017 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.248068 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.248290 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.249129 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.250823 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.251476 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.251526 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.251534 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.251786 (+ 252us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.251832 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:06:57.251893 16242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.247622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33891 (local address 127.7.250.66:56851)
0923 15:06:57.247767 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.247777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.247826 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.248115 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.248116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.248117 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.248207 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.248209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.249271 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.249272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.250711 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.250714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.251599 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.251600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.251695 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.251700 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.251764 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.251806 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260923 15:06:57.252138 16139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c60282ff35b48f080fc7dd11642fafa" candidate_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" is_pre_election: true
I20260923 15:06:57.252264 16139 raft_consensus.cc:2454] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56de0f25a994bd8aa4f57f7afbdd54d in term 0.
I20260923 15:06:57.252373 16220 raft_consensus.cc:504] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.252540 16220 leader_election.cc:288] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [CANDIDATE]: Term 1 election: Requested vote from peers 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457), e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891)
I20260923 15:06:57.252681 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c60282ff35b48f080fc7dd11642fafa" candidate_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
I20260923 15:06:57.252701 16139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c60282ff35b48f080fc7dd11642fafa" candidate_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e105d1ea5a904c47959ce1f02e76c8c2"
I20260923 15:06:57.252759 15857 raft_consensus.cc:3037] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.252771 16139 raft_consensus.cc:3037] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.253424 15857 raft_consensus.cc:2454] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56de0f25a994bd8aa4f57f7afbdd54d in term 1.
I20260923 15:06:57.253593 15928 leader_election.cc:302] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e56de0f25a994bd8aa4f57f7afbdd54d; no voters:
I20260923 15:06:57.253675 16220 raft_consensus.cc:2781] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:57.253667 16139 raft_consensus.cc:2454] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56de0f25a994bd8aa4f57f7afbdd54d in term 1.
I20260923 15:06:57.253813 16220 raft_consensus.cc:686] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 LEADER]: Becoming Leader. State: Replica: e56de0f25a994bd8aa4f57f7afbdd54d, State: Running, Role: LEADER
I20260923 15:06:57.253891 16220 consensus_queue.cc:237] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.254601 15698 catalog_manager.cc:5495] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d reported cstate change: term changed from 0 to 1, leader changed from <none> to e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66). New cstate: current_term: 1 leader_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } health_report { overall_health: UNKNOWN } } }
W20260923 15:06:57.264621 16187 tablet.cc:2369] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176016, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.126@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:57.287209 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.279141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39149 (local address 127.0.0.1:36312)
0923 15:06:57.279365 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.279377 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.279484 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.279666 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.279667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.280037 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:57.280086 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.280088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.280825 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.280826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.281434 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.281436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.281824 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.281825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.281894 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.282537 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:57.282542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:57.284120 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:57.285774 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:57.285783 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:57.285799 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:57.286154 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:57.286450 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:57.286451 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:57.286451 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:57.286530 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:57.286786 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:57.286794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:57.286852 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.286935 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.287067 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260923 15:06:57.287218 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.279298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.7.250.126:39149)
0923 15:06:57.279353 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.279355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.279464 (+ 109us) server_negotiation.cc:407] Connection header received
0923 15:06:57.279580 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.279581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.279595 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.279661 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.280192 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.280736 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.281525 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.281731 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.281823 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.284233 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:57.284234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:57.284235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:57.284242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:57.285685 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.286307 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.286309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.286309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.286373 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.286603 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.286604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.286604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.286714 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:57.286772 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.286983 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.287118 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":31}
I20260923 15:06:57.289283 15698 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:36312:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 15:06:57.289505 15698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:57.291066 16119 tablet_service.cc:1467] Processing CreateTablet for tablet 353c31371e5548eb87a0cb9a6573a89b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30fa9612d4af44f9adfd53b611db1245]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:57.291148 15837 tablet_service.cc:1467] Processing CreateTablet for tablet 353c31371e5548eb87a0cb9a6573a89b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30fa9612d4af44f9adfd53b611db1245]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:57.291237 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353c31371e5548eb87a0cb9a6573a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.291294 15837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353c31371e5548eb87a0cb9a6573a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.292743 16218 tablet_bootstrap.cc:492] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: Bootstrap starting.
I20260923 15:06:57.292814 15978 tablet_service.cc:1467] Processing CreateTablet for tablet 353c31371e5548eb87a0cb9a6573a89b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30fa9612d4af44f9adfd53b611db1245]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:06:57.292956 15978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 353c31371e5548eb87a0cb9a6573a89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.293188 16218 tablet_bootstrap.cc:654] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.293543 16215 tablet_bootstrap.cc:492] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: Bootstrap starting.
I20260923 15:06:57.293902 16215 tablet_bootstrap.cc:654] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.294119 16214 tablet_bootstrap.cc:492] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: Bootstrap starting.
I20260923 15:06:57.294368 16218 tablet_bootstrap.cc:492] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: No bootstrap required, opened a new log
I20260923 15:06:57.294420 16218 ts_tablet_manager.cc:1397] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:06:57.294595 16218 raft_consensus.cc:348] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.294768 16214 tablet_bootstrap.cc:654] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.294800 16218 raft_consensus.cc:374] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.294857 16218 raft_consensus.cc:729] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005c1f4c7adf46f5bf1abfd6b4078bcd, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.294907 16218 consensus_queue.cc:260] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.295024 16218 ts_tablet_manager.cc:1428] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:57.295203 16222 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.295288 16222 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.295624 16214 tablet_bootstrap.cc:492] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: No bootstrap required, opened a new log
I20260923 15:06:57.295696 16214 ts_tablet_manager.cc:1397] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:06:57.295866 16214 raft_consensus.cc:348] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.295953 16214 raft_consensus.cc:374] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.295982 16214 raft_consensus.cc:729] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56de0f25a994bd8aa4f57f7afbdd54d, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.296032 16214 consensus_queue.cc:260] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.296139 16214 ts_tablet_manager.cc:1428] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:57.296283 16220 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.296326 16215 tablet_bootstrap.cc:492] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: No bootstrap required, opened a new log
I20260923 15:06:57.296373 16215 ts_tablet_manager.cc:1397] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 15:06:57.296366 16220 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.296514 16215 raft_consensus.cc:348] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.296593 16215 raft_consensus.cc:374] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.296617 16215 raft_consensus.cc:729] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.296654 16215 consensus_queue.cc:260] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.296752 16215 ts_tablet_manager.cc:1428] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.296762 16221 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.296882 16221 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.473941 16221 raft_consensus.cc:482] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:57.474138 16221 raft_consensus.cc:504] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.474398 16221 leader_election.cc:288] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457), e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265)
I20260923 15:06:57.474643 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353c31371e5548eb87a0cb9a6573a89b" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" is_pre_election: true
I20260923 15:06:57.474771 15857 raft_consensus.cc:2454] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 0.
I20260923 15:06:57.474749 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353c31371e5548eb87a0cb9a6573a89b" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" is_pre_election: true
I20260923 15:06:57.474861 15998 raft_consensus.cc:2454] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 0.
I20260923 15:06:57.475049 16070 leader_election.cc:302] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e105d1ea5a904c47959ce1f02e76c8c2; no voters:
I20260923 15:06:57.475173 16221 raft_consensus.cc:2781] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:57.475225 16221 raft_consensus.cc:482] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:57.475261 16221 raft_consensus.cc:3037] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.475991 16221 raft_consensus.cc:504] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.476168 16221 leader_election.cc:288] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 election: Requested vote from peers 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457), e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265)
I20260923 15:06:57.476385 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353c31371e5548eb87a0cb9a6573a89b" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
I20260923 15:06:57.476408 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "353c31371e5548eb87a0cb9a6573a89b" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
I20260923 15:06:57.476482 15857 raft_consensus.cc:3037] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.476485 15998 raft_consensus.cc:3037] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.477136 15857 raft_consensus.cc:2454] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 1.
I20260923 15:06:57.477339 16070 leader_election.cc:302] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e105d1ea5a904c47959ce1f02e76c8c2; no voters:
I20260923 15:06:57.477331 15998 raft_consensus.cc:2454] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 1.
I20260923 15:06:57.477427 16221 raft_consensus.cc:2781] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:57.477471 16221 raft_consensus.cc:686] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 LEADER]: Becoming Leader. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Running, Role: LEADER
I20260923 15:06:57.477523 16221 consensus_queue.cc:237] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.477779 16224 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: TxnStatusTablet state changed. Reason: New leader e105d1ea5a904c47959ce1f02e76c8c2. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.477860 16224 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:57.477996 16255 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 15:06:57.478127 16255 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 15:06:57.478253 15698 catalog_manager.cc:5495] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } health_report { overall_health: HEALTHY } } }
I20260923 15:06:57.526913 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.524269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.7.250.67:33891)
0923 15:06:57.524331 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.524333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.524341 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:57.524372 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.524373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.524382 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.524433 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.524628 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.525097 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.526185 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.526447 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.526504 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.526698 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.526723 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.526761 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.526839 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.526856 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:06:57.526957 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.524069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33891 (local address 127.0.0.1:33274)
0923 15:06:57.524157 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.524168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.524219 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.524459 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.524460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.524461 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:57.524515 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.524517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.525201 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.525222 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.526098 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.526101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.526529 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.526530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.526599 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.526618 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.526786 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.526791 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.526833 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.526881 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 15:06:57.528200 15857 raft_consensus.cc:1261] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Refusing update from remote peer e105d1ea5a904c47959ce1f02e76c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.528249 15998 raft_consensus.cc:1261] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Refusing update from remote peer e105d1ea5a904c47959ce1f02e76c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.528481 16221 consensus_queue.cc:1035] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.528613 16224 consensus_queue.cc:1035] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.529062 16220 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: TxnStatusTablet state changed. Reason: New leader e105d1ea5a904c47959ce1f02e76c8c2. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.529162 16220 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.529175 16222 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: TxnStatusTablet state changed. Reason: New leader e105d1ea5a904c47959ce1f02e76c8c2. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.529254 16222 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.529966 16224 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.530053 16224 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:57.530088 16260 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.530149 16260 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e105d1ea5a904c47959ce1f02e76c8c2: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:06:57.530570 16222 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.530663 16222 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P 005c1f4c7adf46f5bf1abfd6b4078bcd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.530851 16220 tablet_replica.cc:439] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } } }
I20260923 15:06:57.530937 16220 tablet_replica.cc:442] T 353c31371e5548eb87a0cb9a6573a89b P e56de0f25a994bd8aa4f57f7afbdd54d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:06:57.534523 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.531711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51770 (local address 127.7.250.66:46265)
0923 15:06:57.531752 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.531754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.531802 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:06:57.531866 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.531867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.531887 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.531965 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.532129 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.532842 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.533639 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.533921 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.533998 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.534126 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.534298 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.534383 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.534436 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.534457 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:57.539575 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.537051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:58743 (local address 127.7.250.67:33891)
0923 15:06:57.537094 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.537097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.537235 (+ 138us) server_negotiation.cc:407] Connection header received
0923 15:06:57.537311 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.537312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.537323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.537395 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.537527 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.537976 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.538848 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.539034 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.539140 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.539332 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.539368 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.539427 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.539487 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.539504 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:06:57.539609 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.536944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33891 (local address 127.7.250.66:58743)
0923 15:06:57.537146 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.537164 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.537251 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.537397 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.537398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.537399 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:57.537456 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.537458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.538058 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.538059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.538718 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.538720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.539155 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.539157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.539222 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.539234 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.539429 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.539431 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.539483 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.539526 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":78,"threads_started":1}
I20260923 15:06:57.542081 15857 raft_consensus.cc:1261] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Refusing update from remote peer e56de0f25a994bd8aa4f57f7afbdd54d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.542143 16139 raft_consensus.cc:1261] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Refusing update from remote peer e56de0f25a994bd8aa4f57f7afbdd54d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.542559 16243 consensus_queue.cc:1035] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [LEADER]: Connected to new peer: Peer: permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.542747 16220 consensus_queue.cc:1035] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.551918 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.549068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:41023 (local address 127.7.250.66:46265)
0923 15:06:57.549109 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.549111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.549277 (+ 166us) server_negotiation.cc:407] Connection header received
0923 15:06:57.549337 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.549338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.549347 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.549390 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.549498 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.549963 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.551213 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.551488 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.551544 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.551736 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.551762 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.551795 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.551851 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.551866 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:57.551977 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.549035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46265 (local address 127.7.250.67:41023)
0923 15:06:57.549203 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.549215 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.549278 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.549395 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.549395 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.549397 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:57.549446 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.549448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.550035 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.550036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.551115 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.551118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.551573 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.551574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.551635 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.551651 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.551799 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.551800 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.551849 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.551893 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260923 15:06:57.561928 15698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.562034 15698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36304:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.562773 15698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Sending DeleteTablet for 3 replicas of tablet 0c60282ff35b48f080fc7dd11642fafa
I20260923 15:06:57.563067 16119 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c60282ff35b48f080fc7dd11642fafa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:33262
I20260923 15:06:57.563174 16236 tablet_replica.cc:330] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: stopping tablet replica
I20260923 15:06:57.563234 16236 raft_consensus.cc:2229] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.563272 16236 raft_consensus.cc:2258] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.563273 15978 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c60282ff35b48f080fc7dd11642fafa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:51758
I20260923 15:06:57.563385 16238 tablet_replica.cc:330] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: stopping tablet replica
I20260923 15:06:57.563453 16238 raft_consensus.cc:2229] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:57.563535 16238 raft_consensus.cc:2258] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.563532 15837 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c60282ff35b48f080fc7dd11642fafa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:46044
I20260923 15:06:57.563601 16236 ts_tablet_manager.cc:1905] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.564002 16238 ts_tablet_manager.cc:1905] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.565258 16237 tablet_replica.cc:330] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: stopping tablet replica
I20260923 15:06:57.565320 16237 raft_consensus.cc:2229] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.565369 16237 raft_consensus.cc:2258] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.565547 16238 ts_tablet_manager.cc:1918] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:57.565604 16238 log.cc:1196] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/0c60282ff35b48f080fc7dd11642fafa
I20260923 15:06:57.565694 16236 ts_tablet_manager.cc:1918] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:57.565755 16236 log.cc:1196] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/0c60282ff35b48f080fc7dd11642fafa
I20260923 15:06:57.565871 16238 ts_tablet_manager.cc:1939] T 0c60282ff35b48f080fc7dd11642fafa P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting consensus metadata
I20260923 15:06:57.565994 16236 ts_tablet_manager.cc:1939] T 0c60282ff35b48f080fc7dd11642fafa P e105d1ea5a904c47959ce1f02e76c8c2: Deleting consensus metadata
I20260923 15:06:57.566213 15683 catalog_manager.cc:4841] TS e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265): tablet 0c60282ff35b48f080fc7dd11642fafa (table test-table [id=18c5d13f1e4f46a6977daff66df6a54e]) successfully deleted
I20260923 15:06:57.566390 15684 catalog_manager.cc:4841] TS e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891): tablet 0c60282ff35b48f080fc7dd11642fafa (table test-table [id=18c5d13f1e4f46a6977daff66df6a54e]) successfully deleted
I20260923 15:06:57.566598 16237 ts_tablet_manager.cc:1905] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.568351 16237 ts_tablet_manager.cc:1918] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:57.568413 16237 log.cc:1196] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/0c60282ff35b48f080fc7dd11642fafa
I20260923 15:06:57.568660 16237 ts_tablet_manager.cc:1939] T 0c60282ff35b48f080fc7dd11642fafa P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting consensus metadata
I20260923 15:06:57.569231 15683 catalog_manager.cc:4841] TS 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457): tablet 0c60282ff35b48f080fc7dd11642fafa (table test-table [id=18c5d13f1e4f46a6977daff66df6a54e]) successfully deleted
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:06:57.593811 15750 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:57.586663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.7.250.126:39149)
0923 15:06:57.586734 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.586736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.586859 (+ 123us) server_negotiation.cc:407] Connection header received
0923 15:06:57.586918 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.586919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.586931 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.587005 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.587721 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.588519 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.589588 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.589916 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.589989 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.593737 (+ 3748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36328: BlockingRecv error: recv got EOF from 127.0.0.1:36328 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[15654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM
I20260923 15:06:57.607348 15750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.598595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36334 (local address 127.7.250.126:39149)
0923 15:06:57.598638 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.598640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.598726 (+ 86us) server_negotiation.cc:407] Connection header received
0923 15:06:57.598787 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.598788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.598798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.598864 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:57.599482 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.600261 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.601321 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.601643 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.601737 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.604498 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:57.604500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:57.604501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:57.604509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:57.606096 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.606547 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.606548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.606548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.606608 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:57.606810 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:57.606811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:57.606811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:57.606940 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:57.607032 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.607158 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.607246 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:06:57.609097 15698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:06:57.609268 15698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:57.610683 16119 tablet_service.cc:1467] Processing CreateTablet for tablet 32a54bfdfcdd436896f1c616b8e87d45 (DEFAULT_TABLE table=test-table [id=d1a2303b23a240b38d63b146e453e32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.610773 15837 tablet_service.cc:1467] Processing CreateTablet for tablet 32a54bfdfcdd436896f1c616b8e87d45 (DEFAULT_TABLE table=test-table [id=d1a2303b23a240b38d63b146e453e32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.610831 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a54bfdfcdd436896f1c616b8e87d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.610898 15837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a54bfdfcdd436896f1c616b8e87d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.612121 16215 tablet_bootstrap.cc:492] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Bootstrap starting.
I20260923 15:06:57.612282 15978 tablet_service.cc:1467] Processing CreateTablet for tablet 32a54bfdfcdd436896f1c616b8e87d45 (DEFAULT_TABLE table=test-table [id=d1a2303b23a240b38d63b146e453e32c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.612394 15978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a54bfdfcdd436896f1c616b8e87d45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.612465 16215 tablet_bootstrap.cc:654] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.613262 16215 tablet_bootstrap.cc:492] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: No bootstrap required, opened a new log
I20260923 15:06:57.613312 16215 ts_tablet_manager.cc:1397] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:57.613417 16215 raft_consensus.cc:348] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.613479 16215 raft_consensus.cc:374] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.613499 16215 raft_consensus.cc:729] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.613541 16215 consensus_queue.cc:260] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.613580 16218 tablet_bootstrap.cc:492] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Bootstrap starting.
I20260923 15:06:57.613616 16215 ts_tablet_manager.cc:1428] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.613968 16218 tablet_bootstrap.cc:654] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.614323 16214 tablet_bootstrap.cc:492] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Bootstrap starting.
I20260923 15:06:57.614737 16214 tablet_bootstrap.cc:654] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.615365 16214 tablet_bootstrap.cc:492] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: No bootstrap required, opened a new log
I20260923 15:06:57.615414 16214 ts_tablet_manager.cc:1397] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:57.615541 16214 raft_consensus.cc:348] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.615603 16214 raft_consensus.cc:374] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.615622 16214 raft_consensus.cc:729] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56de0f25a994bd8aa4f57f7afbdd54d, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.615660 16214 consensus_queue.cc:260] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.615734 16218 tablet_bootstrap.cc:492] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: No bootstrap required, opened a new log
I20260923 15:06:57.615738 16214 ts_tablet_manager.cc:1428] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.615777 16218 ts_tablet_manager.cc:1397] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:06:57.615928 16218 raft_consensus.cc:348] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.616003 16218 raft_consensus.cc:374] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.616021 16218 raft_consensus.cc:729] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005c1f4c7adf46f5bf1abfd6b4078bcd, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.616058 16218 consensus_queue.cc:260] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.616130 16218 ts_tablet_manager.cc:1428] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.697768 16222 raft_consensus.cc:482] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:57.697880 16222 raft_consensus.cc:504] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.698335 16222 leader_election.cc:288] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265), e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891)
I20260923 15:06:57.702512 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.698634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:52053 (local address 127.7.250.67:33891)
0923 15:06:57.698703 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.698705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.698767 (+ 62us) server_negotiation.cc:407] Connection header received
0923 15:06:57.698885 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.698886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.698900 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.698972 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.699164 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.699810 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.701218 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.701810 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.701895 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.701905 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.702352 (+ 447us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.702419 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:06:57.702549 16213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.698686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:36415 (local address 127.7.250.66:46265)
0923 15:06:57.698758 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.698760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.698888 (+ 128us) server_negotiation.cc:407] Connection header received
0923 15:06:57.698918 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.698919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.698930 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.698986 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:06:57.699540 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.700225 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.701442 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.702099 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.702240 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.702247 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.702429 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.702469 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 15:06:57.702517 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.698574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46265 (local address 127.7.250.65:36415)
0923 15:06:57.698773 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.698782 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.698837 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.699008 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.699008 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.699011 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.699449 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.699451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.700310 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.700311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.701352 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.701354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.702239 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.702240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.702349 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.702352 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.702404 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.702430 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":21,"thread_start_us":130,"threads_started":1}
I20260923 15:06:57.702517 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.698544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33891 (local address 127.7.250.65:52053)
0923 15:06:57.698699 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.698711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.698799 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.699007 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.699008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.699010 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:06:57.699093 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.699096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.699921 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.699922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.701118 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.701127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.702020 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.702022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.702108 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.702113 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.702343 (+ 230us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.702400 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260923 15:06:57.702993 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a54bfdfcdd436896f1c616b8e87d45" candidate_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" is_pre_election: true
I20260923 15:06:57.703100 15998 raft_consensus.cc:2454] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 005c1f4c7adf46f5bf1abfd6b4078bcd in term 0.
I20260923 15:06:57.703085 16139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a54bfdfcdd436896f1c616b8e87d45" candidate_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" is_pre_election: true
I20260923 15:06:57.703195 16139 raft_consensus.cc:2454] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 005c1f4c7adf46f5bf1abfd6b4078bcd in term 0.
I20260923 15:06:57.703328 15788 leader_election.cc:302] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e56de0f25a994bd8aa4f57f7afbdd54d; no voters:
I20260923 15:06:57.703469 16222 raft_consensus.cc:2781] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:57.703510 16222 raft_consensus.cc:482] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:57.703533 16222 raft_consensus.cc:3037] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.704200 16222 raft_consensus.cc:504] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.704351 16222 leader_election.cc:288] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [CANDIDATE]: Term 1 election: Requested vote from peers e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265), e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891)
I20260923 15:06:57.704541 16139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a54bfdfcdd436896f1c616b8e87d45" candidate_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e105d1ea5a904c47959ce1f02e76c8c2"
I20260923 15:06:57.704541 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a54bfdfcdd436896f1c616b8e87d45" candidate_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
I20260923 15:06:57.704622 16139 raft_consensus.cc:3037] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.704622 15998 raft_consensus.cc:3037] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.705407 16139 raft_consensus.cc:2454] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 005c1f4c7adf46f5bf1abfd6b4078bcd in term 1.
I20260923 15:06:57.705446 15998 raft_consensus.cc:2454] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 005c1f4c7adf46f5bf1abfd6b4078bcd in term 1.
I20260923 15:06:57.705603 15789 leader_election.cc:302] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e105d1ea5a904c47959ce1f02e76c8c2; no voters:
I20260923 15:06:57.705708 16222 raft_consensus.cc:2781] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:57.705893 16222 raft_consensus.cc:686] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 LEADER]: Becoming Leader. State: Replica: 005c1f4c7adf46f5bf1abfd6b4078bcd, State: Running, Role: LEADER
I20260923 15:06:57.705993 16222 consensus_queue.cc:237] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.706739 15699 catalog_manager.cc:5495] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd reported cstate change: term changed from 0 to 1, leader changed from <none> to 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65). New cstate: current_term: 1 leader_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } health_report { overall_health: UNKNOWN } } }
I20260923 15:06:57.712845 16212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.710026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46056 (local address 127.7.250.65:34457)
0923 15:06:57.710070 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.710073 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.710132 (+ 59us) server_negotiation.cc:407] Connection header received
0923 15:06:57.710347 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.710348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.710369 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.710452 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.710626 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.711139 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.711978 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.712195 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.712276 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.712473 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.712588 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.712691 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.712734 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.712757 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:57.713866 15998 raft_consensus.cc:1261] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Refusing update from remote peer 005c1f4c7adf46f5bf1abfd6b4078bcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.713912 16139 raft_consensus.cc:1261] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Refusing update from remote peer 005c1f4c7adf46f5bf1abfd6b4078bcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.714291 16288 consensus_queue.cc:1035] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.714442 16222 consensus_queue.cc:1035] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.716794 16227 mvcc.cc:205] Tried to move back new op lower bound from 7332560968553914368 to 7332560968524410880. Current Snapshot: MvccSnapshot[applied={T|T < 7332560968553914368 or (T in {7332560968553914368})}]
I20260923 15:06:57.718540 15699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.718636 15699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.719424 15699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Sending DeleteTablet for 3 replicas of tablet 32a54bfdfcdd436896f1c616b8e87d45
I20260923 15:06:57.719676 16119 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a54bfdfcdd436896f1c616b8e87d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:33262
I20260923 15:06:57.719692 15978 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a54bfdfcdd436896f1c616b8e87d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:51758
I20260923 15:06:57.719708 15837 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a54bfdfcdd436896f1c616b8e87d45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:46044
I20260923 15:06:57.719800 16236 tablet_replica.cc:330] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: stopping tablet replica
I20260923 15:06:57.719868 16236 raft_consensus.cc:2229] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.719920 16236 raft_consensus.cc:2258] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.719996 16237 tablet_replica.cc:330] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: stopping tablet replica
I20260923 15:06:57.720052 16237 raft_consensus.cc:2229] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:57.720134 16237 raft_consensus.cc:2258] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.720238 16238 tablet_replica.cc:330] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: stopping tablet replica
I20260923 15:06:57.720180 15699 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 15:06:57.720297 16238 raft_consensus.cc:2229] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus shutting down.
W20260923 15:06:57.720322 15699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:06:57.720346 16238 raft_consensus.cc:2258] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.720707 16236 ts_tablet_manager.cc:1905] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.721545 15837 tablet_service.cc:1467] Processing CreateTablet for tablet 5819219f0f9d47f7a52142a9b4ed32a8 (DEFAULT_TABLE table=test-table [id=a38c2dc070444dedb196d8357daaba8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.721625 16119 tablet_service.cc:1467] Processing CreateTablet for tablet 5819219f0f9d47f7a52142a9b4ed32a8 (DEFAULT_TABLE table=test-table [id=a38c2dc070444dedb196d8357daaba8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.721659 15837 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5819219f0f9d47f7a52142a9b4ed32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.721736 16119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5819219f0f9d47f7a52142a9b4ed32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.722007 16237 ts_tablet_manager.cc:1905] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.723274 16218 tablet_bootstrap.cc:492] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Bootstrap starting.
I20260923 15:06:57.723673 16237 ts_tablet_manager.cc:1918] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:57.723727 16218 tablet_bootstrap.cc:654] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.723742 16237 log.cc:1196] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/32a54bfdfcdd436896f1c616b8e87d45
I20260923 15:06:57.723979 16237 ts_tablet_manager.cc:1939] T 32a54bfdfcdd436896f1c616b8e87d45 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting consensus metadata
I20260923 15:06:57.724069 16215 tablet_bootstrap.cc:492] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Bootstrap starting.
I20260923 15:06:57.724151 15978 tablet_service.cc:1467] Processing CreateTablet for tablet 5819219f0f9d47f7a52142a9b4ed32a8 (DEFAULT_TABLE table=test-table [id=a38c2dc070444dedb196d8357daaba8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:06:57.724295 15978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5819219f0f9d47f7a52142a9b4ed32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:57.724428 16215 tablet_bootstrap.cc:654] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.725020 16215 tablet_bootstrap.cc:492] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: No bootstrap required, opened a new log
I20260923 15:06:57.725071 16215 ts_tablet_manager.cc:1397] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:57.725203 16215 raft_consensus.cc:348] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.725273 16215 raft_consensus.cc:374] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.725293 16215 raft_consensus.cc:729] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.725339 16215 consensus_queue.cc:260] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.725423 16215 ts_tablet_manager.cc:1428] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:57.725807 16214 tablet_bootstrap.cc:492] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Bootstrap starting.
I20260923 15:06:57.726125 16214 tablet_bootstrap.cc:654] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:57.726574 16236 ts_tablet_manager.cc:1918] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:57.726625 16236 log.cc:1196] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/32a54bfdfcdd436896f1c616b8e87d45
I20260923 15:06:57.726819 16236 ts_tablet_manager.cc:1939] T 32a54bfdfcdd436896f1c616b8e87d45 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting consensus metadata
I20260923 15:06:57.727373 15683 catalog_manager.cc:4841] TS 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457): tablet 32a54bfdfcdd436896f1c616b8e87d45 (table test-table [id=d1a2303b23a240b38d63b146e453e32c]) successfully deleted
I20260923 15:06:57.727440 15684 catalog_manager.cc:4841] TS e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891): tablet 32a54bfdfcdd436896f1c616b8e87d45 (table test-table [id=d1a2303b23a240b38d63b146e453e32c]) successfully deleted
I20260923 15:06:57.727692 16238 ts_tablet_manager.cc:1905] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.727970 16218 tablet_bootstrap.cc:492] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: No bootstrap required, opened a new log
I20260923 15:06:57.728012 16218 ts_tablet_manager.cc:1397] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 15:06:57.728168 16218 raft_consensus.cc:348] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.728237 16218 raft_consensus.cc:374] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.728258 16218 raft_consensus.cc:729] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005c1f4c7adf46f5bf1abfd6b4078bcd, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.728319 16218 consensus_queue.cc:260] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.728433 16218 ts_tablet_manager.cc:1428] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.728878 16214 tablet_bootstrap.cc:492] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: No bootstrap required, opened a new log
I20260923 15:06:57.728920 16214 ts_tablet_manager.cc:1397] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 15:06:57.729063 16214 raft_consensus.cc:348] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.729143 16214 raft_consensus.cc:374] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:57.729172 16214 raft_consensus.cc:729] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56de0f25a994bd8aa4f57f7afbdd54d, State: Initialized, Role: FOLLOWER
I20260923 15:06:57.729233 16214 consensus_queue.cc:260] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.729281 16238 ts_tablet_manager.cc:1918] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:06:57.729326 16238 log.cc:1196] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/32a54bfdfcdd436896f1c616b8e87d45
I20260923 15:06:57.729346 16214 ts_tablet_manager.cc:1428] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:57.729547 16238 ts_tablet_manager.cc:1939] T 32a54bfdfcdd436896f1c616b8e87d45 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting consensus metadata
I20260923 15:06:57.729951 15683 catalog_manager.cc:4841] TS e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265): tablet 32a54bfdfcdd436896f1c616b8e87d45 (table test-table [id=d1a2303b23a240b38d63b146e453e32c]) successfully deleted
I20260923 15:06:57.811110 16224 raft_consensus.cc:482] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:06:57.811237 16224 raft_consensus.cc:504] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.811434 16224 leader_election.cc:288] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457), e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265)
I20260923 15:06:57.811653 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5819219f0f9d47f7a52142a9b4ed32a8" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" is_pre_election: true
I20260923 15:06:57.811672 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5819219f0f9d47f7a52142a9b4ed32a8" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" is_pre_election: true
I20260923 15:06:57.811772 15857 raft_consensus.cc:2454] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 0.
I20260923 15:06:57.811782 15998 raft_consensus.cc:2454] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 0.
I20260923 15:06:57.811936 16070 leader_election.cc:302] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e105d1ea5a904c47959ce1f02e76c8c2, e56de0f25a994bd8aa4f57f7afbdd54d; no voters:
I20260923 15:06:57.812081 16224 raft_consensus.cc:2781] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:06:57.812230 16224 raft_consensus.cc:482] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:06:57.812275 16224 raft_consensus.cc:3037] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.813319 16224 raft_consensus.cc:504] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.813462 16224 leader_election.cc:288] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 election: Requested vote from peers 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457), e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265)
I20260923 15:06:57.813652 15857 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5819219f0f9d47f7a52142a9b4ed32a8" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd"
I20260923 15:06:57.813684 15998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5819219f0f9d47f7a52142a9b4ed32a8" candidate_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d"
I20260923 15:06:57.813757 15857 raft_consensus.cc:3037] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.813764 15998 raft_consensus.cc:3037] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:57.814520 15857 raft_consensus.cc:2454] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 1.
I20260923 15:06:57.814589 15998 raft_consensus.cc:2454] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e105d1ea5a904c47959ce1f02e76c8c2 in term 1.
I20260923 15:06:57.814723 16070 leader_election.cc:302] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 005c1f4c7adf46f5bf1abfd6b4078bcd, e105d1ea5a904c47959ce1f02e76c8c2; no voters:
I20260923 15:06:57.814852 16224 raft_consensus.cc:2781] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:57.814910 16224 raft_consensus.cc:686] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 LEADER]: Becoming Leader. State: Replica: e105d1ea5a904c47959ce1f02e76c8c2, State: Running, Role: LEADER
I20260923 15:06:57.814960 16224 consensus_queue.cc:237] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } }
I20260923 15:06:57.815517 15699 catalog_manager.cc:5495] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e105d1ea5a904c47959ce1f02e76c8c2" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33891 } health_report { overall_health: HEALTHY } } }
I20260923 15:06:57.845295 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.842775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33290 (local address 127.7.250.67:33891)
0923 15:06:57.842841 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.842844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.842853 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:57.842922 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.842923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.842936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.842988 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.843094 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.843721 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.844584 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.844795 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.844879 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.845020 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.845052 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.845100 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.845175 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.845198 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:06:57.849391 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.847574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:55027 (local address 127.7.250.67:33891)
0923 15:06:57.847608 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:57.847610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:57.847616 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:06:57.847632 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:57.847633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:57.847641 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:57.847683 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:57.847758 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.848177 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.848858 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.849021 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.849069 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.849213 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.849233 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.849268 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:57.849301 (+ 33us) server_negotiation.cc:299] Negotiation successful
0923 15:06:57.849325 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":17}
I20260923 15:06:57.849395 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.847449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33891 (local address 127.7.250.67:55027)
0923 15:06:57.847514 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.847524 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:57.847567 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:57.847692 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:57.847693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:57.847694 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:06:57.847723 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.847724 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.848239 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.848239 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.848786 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:57.848788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:57.849087 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:57.849088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:57.849125 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:57.849130 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:06:57.849269 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:06:57.849270 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:06:57.849300 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 15:06:57.849339 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":2}
I20260923 15:06:57.851454 15998 raft_consensus.cc:1261] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Refusing update from remote peer e105d1ea5a904c47959ce1f02e76c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.851490 15857 raft_consensus.cc:1261] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Refusing update from remote peer e105d1ea5a904c47959ce1f02e76c8c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:57.851699 16221 consensus_queue.cc:1035] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "005c1f4c7adf46f5bf1abfd6b4078bcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 34457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.851809 16260 consensus_queue.cc:1035] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56de0f25a994bd8aa4f57f7afbdd54d" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:57.861702 15699 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.861792 15699 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:06:57.862430 15699 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 829ea8f74a6a4d6b86be2ea0b382e1c9: Sending DeleteTablet for 3 replicas of tablet 5819219f0f9d47f7a52142a9b4ed32a8
I20260923 15:06:57.862725 16119 tablet_service.cc:1514] Processing DeleteTablet for tablet 5819219f0f9d47f7a52142a9b4ed32a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:33262
I20260923 15:06:57.862741 15978 tablet_service.cc:1514] Processing DeleteTablet for tablet 5819219f0f9d47f7a52142a9b4ed32a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:51758
I20260923 15:06:57.862747 15837 tablet_service.cc:1514] Processing DeleteTablet for tablet 5819219f0f9d47f7a52142a9b4ed32a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:06:57 UTC) from {username='oryx', principal='oryx/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:46044
I20260923 15:06:57.862824 16236 tablet_replica.cc:330] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: stopping tablet replica
I20260923 15:06:57.862824 16238 tablet_replica.cc:330] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: stopping tablet replica
I20260923 15:06:57.862840 16237 tablet_replica.cc:330] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: stopping tablet replica
I20260923 15:06:57.862870 16238 raft_consensus.cc:2229] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.862874 16236 raft_consensus.cc:2229] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:06:57.862879 16237 raft_consensus.cc:2229] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:06:57.862907 16238 raft_consensus.cc:2258] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.862913 16237 raft_consensus.cc:2258] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.862934 16236 raft_consensus.cc:2258] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:06:57.863231 16236 ts_tablet_manager.cc:1905] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.863234 16238 ts_tablet_manager.cc:1905] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.863286 16237 ts_tablet_manager.cc:1905] T 5819219f0f9d47f7a52142a9b4ed32a8 P 005c1f4c7adf46f5bf1abfd6b4078bcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:06:57.864784 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 15772
I20260923 15:06:57.864928 16238 ts_tablet_manager.cc:1918] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:57.864980 16238 log.cc:1196] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/5819219f0f9d47f7a52142a9b4ed32a8
I20260923 15:06:57.865253 16238 ts_tablet_manager.cc:1939] T 5819219f0f9d47f7a52142a9b4ed32a8 P e56de0f25a994bd8aa4f57f7afbdd54d: Deleting consensus metadata
I20260923 15:06:57.865926 16236 ts_tablet_manager.cc:1918] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:06:57.865993 16236 log.cc:1196] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/5819219f0f9d47f7a52142a9b4ed32a8
I20260923 15:06:57.866210 16236 ts_tablet_manager.cc:1939] T 5819219f0f9d47f7a52142a9b4ed32a8 P e105d1ea5a904c47959ce1f02e76c8c2: Deleting consensus metadata
I20260923 15:06:57.866555 15684 catalog_manager.cc:4841] TS e105d1ea5a904c47959ce1f02e76c8c2 (127.7.250.67:33891): tablet 5819219f0f9d47f7a52142a9b4ed32a8 (table test-table [id=a38c2dc070444dedb196d8357daaba8e]) successfully deleted
I20260923 15:06:57.868221 15683 catalog_manager.cc:4841] TS e56de0f25a994bd8aa4f57f7afbdd54d (127.7.250.66:46265): tablet 5819219f0f9d47f7a52142a9b4ed32a8 (table test-table [id=a38c2dc070444dedb196d8357daaba8e]) successfully deleted
W20260923 15:06:57.873068 15683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:34457 (error 108)
I20260923 15:06:57.873330 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 15912
I20260923 15:06:57.875177 16303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:57.873477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:34457 (local address 127.0.0.1:46072)
0923 15:06:57.875034 (+ 1557us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:57.875080 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:34457: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1540,"thread_start_us":43,"threads_started":1}
W20260923 15:06:57.875308 15683 catalog_manager.cc:4584] TS 005c1f4c7adf46f5bf1abfd6b4078bcd (127.7.250.65:34457): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5819219f0f9d47f7a52142a9b4ed32a8: Network error: Client connection negotiation failed: client connection to 127.7.250.65:34457: connect: Connection refused (error 111)
I20260923 15:06:57.879377 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16054
I20260923 15:06:57.885063 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 15667
2026-09-23T15:06:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2725 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[16306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[16306](info): set up 2 sockets
Sep 23 15:06:57 dist-test-slave-bb3x krb5kdc[16306](info): commencing operation
krb5kdc: starting...
W20260923 15:06:58.932433 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:06:59 dist-test-slave-bb3x krb5kdc[16306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:06:59.055238 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45597
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:45597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:59.139882 16320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:59.140087 16320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:59.140110 16320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:59.141281 16320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:59.141325 16320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:59.141340 16320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:59.141352 16320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:59.141402 16320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:59.142935 16320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:45597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:59.143195 16320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:59.143464 16320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:59.148948 16320 server_base.cc:1034] running on GCE node
W20260923 15:06:59.148921 16326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:59.149083 16328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:59.148916 16325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:59.149643 16320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:59.150086 16320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:59.151280 16320 hybrid_clock.cc:648] HybridClock initialized: now 1790176019151251 us; error 40 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T15:06:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1267 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:59 dist-test-slave-bb3x krb5kdc[16333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:59 dist-test-slave-bb3x krb5kdc[16333](info): set up 2 sockets
Sep 23 15:06:59 dist-test-slave-bb3x krb5kdc[16333](info): commencing operation
krb5kdc: starting...
W20260923 15:07:00.248768 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:07:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:07:00.374744 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41533
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:00.456327 16346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:00.456580 16346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:00.456619 16346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:00.457793 16346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:07:00.457864 16346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:00.457887 16346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:00.457908 16346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:07:00.457937 16346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:07:00.459508 16346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:00.459797 16346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:00.460026 16346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:00.465025 16351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:00.465193 16346 server_base.cc:1034] running on GCE node
W20260923 15:07:00.465145 16354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:00.465020 16352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:00.465673 16346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:00.466032 16346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:00.467211 16346 hybrid_clock.cc:648] HybridClock initialized: now 1790176020467194 us; error 27 us; skew 500 ppm
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:00.470079 16346 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:07:00.470636 16346 webserver.cc:460] Webserver started at http://127.7.250.126:39259/ using document root <none> and password file <none>
I20260923 15:07:00.470906 16346 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:00.470956 16346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:00.471087 16346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:00.472294 16346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "267036c7d28a43e298a39008ed1d73c9"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "ad6342f70b07f5c80b9eea1992166e1d"
server_key_iv: "e7655e5c6c484393426a113a6ca59df5"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.472681 16346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "267036c7d28a43e298a39008ed1d73c9"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "ad6342f70b07f5c80b9eea1992166e1d"
server_key_iv: "e7655e5c6c484393426a113a6ca59df5"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.474206 16346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:07:00.475232 16361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.475451 16346 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:00.475572 16346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "267036c7d28a43e298a39008ed1d73c9"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "ad6342f70b07f5c80b9eea1992166e1d"
server_key_iv: "e7655e5c6c484393426a113a6ca59df5"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.475664 16346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:00.497812 16346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:00.500363 16346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:00.500526 16346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:00.505021 16346 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41533
I20260923 15:07:00.505088 16413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41533 every 8 connection(s)
I20260923 15:07:00.505481 16346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:07:00.506618 16414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:00.509192 16414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Bootstrap starting.
I20260923 15:07:00.509805 16414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:00.509964 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16346
I20260923 15:07:00.510176 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:07:00.510236 16414 log.cc:824] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:00.510430 8169 external_mini_cluster.cc:1428] Setting key 874968dd212ddfe221b4c033b83c4437
I20260923 15:07:00.511265 16414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: No bootstrap required, opened a new log
I20260923 15:07:00.514197 16414 raft_consensus.cc:348] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:00.514425 16414 raft_consensus.cc:374] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:00.514461 16414 raft_consensus.cc:729] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267036c7d28a43e298a39008ed1d73c9, State: Initialized, Role: FOLLOWER
I20260923 15:07:00.514599 16414 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:00.514693 16414 raft_consensus.cc:388] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:07:00.514742 16414 raft_consensus.cc:482] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:07:00.514797 16414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:00.515456 16414 raft_consensus.cc:504] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:00.515570 16414 leader_election.cc:302] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267036c7d28a43e298a39008ed1d73c9; no voters:
I20260923 15:07:00.515743 16414 leader_election.cc:288] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:07:00.515806 16419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:07:00.515964 16419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 LEADER]: Becoming Leader. State: Replica: 267036c7d28a43e298a39008ed1d73c9, State: Running, Role: LEADER
I20260923 15:07:00.516029 16414 sys_catalog.cc:564] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:07:00.516108 16419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:00.516495 16419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267036c7d28a43e298a39008ed1d73c9. Latest consensus state: current_term: 1 leader_uuid: "267036c7d28a43e298a39008ed1d73c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } } }
I20260923 15:07:00.516691 16421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "267036c7d28a43e298a39008ed1d73c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } } }
I20260923 15:07:00.516970 16419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:00.517083 16421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:00.517256 16428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:07:00.517894 16428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:00.520236 16428 catalog_manager.cc:1347] Generated new cluster ID: 7a69470caf77484bba6f5a60cf688826
I20260923 15:07:00.520303 16428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:07:00.522773 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:00.510983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59836 (local address 127.7.250.126:41533)
0923 15:07:00.511186 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:00.511197 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:00.511237 (+ 40us) server_negotiation.cc:407] Connection header received
0923 15:07:00.511822 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:00.511833 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:00.511904 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:00.512073 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:00.512839 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.514519 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.515455 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.516193 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.516365 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:00.519224 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:00.519229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:00.519229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:00.519243 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:00.521190 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:00.521586 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:00.521589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:00.521590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:00.521683 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:00.521882 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:00.521883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:00.521884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:00.522139 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:00.522328 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:00.522446 (+ 118us) server_negotiation.cc:299] Negotiation successful
0923 15:07:00.522592 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260923 15:07:00.552659 16428 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:07:00.553316 16428 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:07:00.561386 16428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Generated new TSK 0
I20260923 15:07:00.561597 16428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:07:00.598133 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41533
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:00.709224 16440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:00.709491 16440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:00.709538 16440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:00.710907 16440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:00.710980 16440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:00.711032 16440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:07:00.712684 16440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:00.712984 16440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:00.713227 16440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:00.718935 16446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:00.718930 16445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:00.719010 16440 server_base.cc:1034] running on GCE node
W20260923 15:07:00.718930 16448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:00.719381 16440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:00.719607 16440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:00.720788 16440 hybrid_clock.cc:648] HybridClock initialized: now 1790176020720772 us; error 30 us; skew 500 ppm
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:00.723644 16440 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:07:00.724185 16440 webserver.cc:460] Webserver started at http://127.7.250.65:32913/ using document root <none> and password file <none>
I20260923 15:07:00.724402 16440 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:00.724448 16440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:00.724548 16440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:00.725658 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "962ba3c027504b75b5c09b88eacdc73d"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "d16fcb7cc73aed176563ad3857228490"
server_key_iv: "84f2a0b215fbeebe3ab939b05a757f1a"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.726029 16440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "962ba3c027504b75b5c09b88eacdc73d"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "d16fcb7cc73aed176563ad3857228490"
server_key_iv: "84f2a0b215fbeebe3ab939b05a757f1a"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.727617 16440 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:07:00.728523 16455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.728699 16440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:00.728834 16440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "962ba3c027504b75b5c09b88eacdc73d"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "d16fcb7cc73aed176563ad3857228490"
server_key_iv: "84f2a0b215fbeebe3ab939b05a757f1a"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.728896 16440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:00.746423 16440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:00.748931 16440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:00.749059 16440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:00.749264 16440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:00.749536 16440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:00.749569 16440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.749594 16440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:00.749608 16440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.756170 16440 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:36181
I20260923 15:07:00.756304 16569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:36181 every 8 connection(s)
I20260923 15:07:00.756652 16440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:00.764482 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16440
I20260923 15:07:00.764636 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:07:00.764950 8169 external_mini_cluster.cc:1428] Setting key fb45e156ed10c73d4f4987127d08aeba
I20260923 15:07:00.767409 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:00.757563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:44343 (local address 127.7.250.126:41533)
0923 15:07:00.757637 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:00.757640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:00.757882 (+ 242us) server_negotiation.cc:407] Connection header received
0923 15:07:00.758429 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:00.758430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:00.758442 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:00.758489 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:00.759512 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.760081 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.761054 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.761277 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.761347 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:00.763908 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:00.763910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:00.763910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:00.763917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:00.765819 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:00.766332 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:00.766334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:00.766334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:00.766397 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:00.766699 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:00.766700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:00.766701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:00.766826 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:00.766905 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:00.767196 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 15:07:00.767310 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:07:00.767617 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:00.757522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:44343)
0923 15:07:00.757726 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:00.757765 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:00.758323 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:00.758543 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:00.758546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:00.759224 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:00.759427 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:00.759432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.760177 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.760178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:00.760976 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:00.760978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.761389 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.761390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:00.761527 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:00.762072 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:00.762085 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:00.763809 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:00.765932 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:00.765945 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:00.765948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:00.766205 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:00.766478 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:00.766479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:00.766479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:00.766621 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:00.766896 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:00.766903 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:00.767049 (+ 146us) client_negotiation.cc:769] Sending connection context
0923 15:07:00.767160 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:07:00.767349 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":115,"threads_started":1}
I20260923 15:07:00.768169 16570 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41533
I20260923 15:07:00.768301 16570 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:00.768504 16570 heartbeater.cc:511] Master 127.7.250.126:41533 requested a full tablet report, sending...
I20260923 15:07:00.768996 16378 ts_manager.cc:194] Registered new tserver with Master: 962ba3c027504b75b5c09b88eacdc73d (127.7.250.65:36181)
I20260923 15:07:00.769657 16378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:44343
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:07:00.793881 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41533
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:00.874991 16575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:00.875223 16575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:00.875264 16575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:00.876458 16575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:00.876523 16575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:00.876572 16575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:07:00.878031 16575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:00.878376 16575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:00.878614 16575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:00.883903 16580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:00.883908 16581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:00.884146 16575 server_base.cc:1034] running on GCE node
W20260923 15:07:00.883908 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:00.884475 16575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:00.884733 16575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:00.885946 16575 hybrid_clock.cc:648] HybridClock initialized: now 1790176020885935 us; error 35 us; skew 500 ppm
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:00.889006 16575 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:07:00.889570 16575 webserver.cc:460] Webserver started at http://127.7.250.66:42949/ using document root <none> and password file <none>
I20260923 15:07:00.889827 16575 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:00.889894 16575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:00.890036 16575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:00.891150 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "89762907fd5c429fb93eb47d0dd41162"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "774f55c71e30f8ec723b499fee948aa5"
server_key_iv: "c454a2b21dee6456106e022607213bf3"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.891508 16575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "89762907fd5c429fb93eb47d0dd41162"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "774f55c71e30f8ec723b499fee948aa5"
server_key_iv: "c454a2b21dee6456106e022607213bf3"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.892901 16575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:07:00.893764 16590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.893958 16575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:00.894057 16575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "89762907fd5c429fb93eb47d0dd41162"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "774f55c71e30f8ec723b499fee948aa5"
server_key_iv: "c454a2b21dee6456106e022607213bf3"
server_key_version: "encryptionkey@0"
I20260923 15:07:00.894179 16575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:00.905685 16575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:00.907809 16575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:00.907989 16575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:00.908233 16575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:00.908547 16575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:00.908597 16575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.908639 16575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:00.908670 16575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:00.914547 16575 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:35025
I20260923 15:07:00.914652 16704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:35025 every 8 connection(s)
I20260923 15:07:00.914973 16575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:07:00.919492 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16575
I20260923 15:07:00.919731 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:07:00.919975 8169 external_mini_cluster.cc:1428] Setting key 5d657fed341ad2c6581163b5c4bea08f
Sep 23 15:07:00 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176020, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:00.925827 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:00.915717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:46813 (local address 127.7.250.126:41533)
0923 15:07:00.915761 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:00.915763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:00.915991 (+ 228us) server_negotiation.cc:407] Connection header received
0923 15:07:00.916629 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:00.916631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:00.916644 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:00.916697 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:00.917694 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.918209 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.919255 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.919461 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.919533 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:00.922161 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:00.922162 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:00.922163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:00.922171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:00.924078 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:00.924626 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:00.924627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:00.924628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:00.924697 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:00.925045 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:00.925046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:00.925046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:00.925220 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:00.925296 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:00.925588 (+ 292us) server_negotiation.cc:299] Negotiation successful
0923 15:07:00.925680 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:07:00.925906 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:00.915701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:46813)
0923 15:07:00.915865 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:00.915903 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:00.916480 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:00.916777 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:00.916781 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:00.917355 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:00.917599 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:00.917605 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.918339 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.918340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:00.919149 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:00.919151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:00.919546 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:00.919547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:00.919683 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:00.920353 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:00.920364 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:00.922045 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:00.924251 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:00.924258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:00.924260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:00.924546 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:00.924795 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:00.924796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:00.924796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:00.924959 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:00.925306 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:00.925312 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:00.925439 (+ 127us) client_negotiation.cc:769] Sending connection context
0923 15:07:00.925532 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 15:07:00.925694 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260923 15:07:00.926483 16705 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41533
I20260923 15:07:00.926604 16705 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:00.926831 16705 heartbeater.cc:511] Master 127.7.250.126:41533 requested a full tablet report, sending...
I20260923 15:07:00.927275 16378 ts_manager.cc:194] Registered new tserver with Master: 89762907fd5c429fb93eb47d0dd41162 (127.7.250.66:35025)
I20260923 15:07:00.927673 16378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:46813
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:07:00.949926 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41533
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:01.034461 16710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:01.034714 16710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:01.034755 16710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:01.035979 16710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:01.036043 16710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:01.036090 16710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:07:01.037581 16710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:01.037853 16710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:01.038087 16710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:01.043704 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:01.043731 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:01.043695 16715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.043936 16710 server_base.cc:1034] running on GCE node
I20260923 15:07:01.044245 16710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:01.044492 16710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:01.045650 16710 hybrid_clock.cc:648] HybridClock initialized: now 1790176021045627 us; error 37 us; skew 500 ppm
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176021, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:01.048503 16710 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:07:01.048954 16710 webserver.cc:460] Webserver started at http://127.7.250.67:40691/ using document root <none> and password file <none>
I20260923 15:07:01.049196 16710 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:01.049279 16710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:01.049427 16710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:01.050601 16710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "3adf5e5a7f1a48d59aa6b04dfb701b59"
format_stamp: "Formatted at 2026-09-23 15:07:01 on dist-test-slave-bb3x"
server_key: "59d89bc3c3f2fd44ab64d30a8af5297d"
server_key_iv: "844276b508e280312b10fcfbd73e67e3"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.050958 16710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "3adf5e5a7f1a48d59aa6b04dfb701b59"
format_stamp: "Formatted at 2026-09-23 15:07:01 on dist-test-slave-bb3x"
server_key: "59d89bc3c3f2fd44ab64d30a8af5297d"
server_key_iv: "844276b508e280312b10fcfbd73e67e3"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.052480 16710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:01.053490 16725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.053720 16710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:01.053844 16710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "3adf5e5a7f1a48d59aa6b04dfb701b59"
format_stamp: "Formatted at 2026-09-23 15:07:01 on dist-test-slave-bb3x"
server_key: "59d89bc3c3f2fd44ab64d30a8af5297d"
server_key_iv: "844276b508e280312b10fcfbd73e67e3"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.053927 16710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:01.069617 16710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:01.072228 16710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:01.072419 16710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:01.072674 16710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:01.073002 16710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:01.073065 16710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.073130 16710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:01.073164 16710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.079638 16710 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:38779
I20260923 15:07:01.079782 16839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:38779 every 8 connection(s)
I20260923 15:07:01.080092 16710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:07:01.085182 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16710
I20260923 15:07:01.085287 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:07:01.085489 8169 external_mini_cluster.cc:1428] Setting key 73f2b1e9e9d8d76e814ef920a0df0357
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176021, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:01.090942 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.080892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:50229 (local address 127.7.250.126:41533)
0923 15:07:01.080985 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.080988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.081194 (+ 206us) server_negotiation.cc:407] Connection header received
0923 15:07:01.081801 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.081803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.081817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.081877 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.083095 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.083678 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.084797 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.085035 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.085147 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.087898 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:01.087899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:01.087899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:01.087907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:01.089438 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:01.089942 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:01.089943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:01.089943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:01.090008 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:01.090273 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:01.090274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:01.090274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:01.090396 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:01.090470 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:01.090760 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 15:07:01.090845 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:07:01.091176 16842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.080812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:50229)
0923 15:07:01.081052 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.081100 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.081680 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.081973 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.081978 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.082751 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.082970 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.082977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.083781 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.083783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.084675 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.084678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.085159 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.085161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.085352 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.086030 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.086042 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.087787 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:01.089520 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:01.089526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:01.089528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:01.089830 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:01.090074 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:01.090075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:01.090075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:01.090180 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:01.090461 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:01.090467 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:01.090604 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 15:07:01.090739 (+ 135us) client_negotiation.cc:240] Negotiation successful
0923 15:07:01.090906 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260923 15:07:01.091706 16840 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41533
I20260923 15:07:01.091830 16840 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:01.092006 16840 heartbeater.cc:511] Master 127.7.250.126:41533 requested a full tablet report, sending...
I20260923 15:07:01.092451 16378 ts_manager.cc:194] Registered new tserver with Master: 3adf5e5a7f1a48d59aa6b04dfb701b59 (127.7.250.67:38779)
I20260923 15:07:01.093092 16378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:50229
I20260923 15:07:01.098052 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab.
Entry for principal oryx/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab.
WARNING: no policy specified for oryx/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab.
Entry for principal oryx/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab.
WARNING: no policy specified for oryx/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab.
Entry for principal oryx/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab.
I20260923 15:07:01.178934 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16440
I20260923 15:07:01.184176 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16575
I20260923 15:07:01.188711 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16710
I20260923 15:07:01.193341 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16346
I20260923 15:07:01.199051 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41533
--webserver_interface=127.7.250.126
--webserver_port=39259
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:01.288373 16849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:01.288637 16849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:01.288681 16849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:01.289958 16849 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:07:01.290024 16849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:01.290061 16849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:01.290093 16849 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:07:01.290122 16849 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:07:01.291904 16849 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:01.292227 16849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:01.292490 16849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:01.297981 16854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:01.297992 16857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.298332 16849 server_base.cc:1034] running on GCE node
W20260923 15:07:01.298085 16855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.298774 16849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:01.299095 16849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:01.300290 16849 hybrid_clock.cc:648] HybridClock initialized: now 1790176021300263 us; error 39 us; skew 500 ppm
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176021, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:01.303462 16849 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:07:01.303977 16849 webserver.cc:460] Webserver started at http://127.7.250.126:39259/ using document root <none> and password file <none>
I20260923 15:07:01.304234 16849 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:01.304330 16849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:01.305909 16849 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:01.306775 16864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.306985 16849 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:07:01.307188 16849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "267036c7d28a43e298a39008ed1d73c9"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "ad6342f70b07f5c80b9eea1992166e1d"
server_key_iv: "e7655e5c6c484393426a113a6ca59df5"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.307579 16849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:01.334971 16849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:01.337205 16849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:01.337380 16849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:01.341444 16849 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41533
I20260923 15:07:01.341480 16917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41533 every 8 connection(s)
I20260923 15:07:01.341892 16849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:07:01.343422 16918 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:07:01.344100 16918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Bootstrap starting.
I20260923 15:07:01.344419 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16849
I20260923 15:07:01.344707 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:36181
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:41533
--webserver_port=32913
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:07:01.353327 16918 log.cc:824] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:01.355685 16918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:07:01.356034 16918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Bootstrap complete.
I20260923 15:07:01.360123 16918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:01.360445 16918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 267036c7d28a43e298a39008ed1d73c9, State: Initialized, Role: FOLLOWER
I20260923 15:07:01.360661 16918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:01.360737 16918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:07:01.360785 16918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:07:01.360832 16918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:07:01.362178 16918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:01.362313 16918 leader_election.cc:302] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 267036c7d28a43e298a39008ed1d73c9; no voters:
I20260923 15:07:01.362478 16918 leader_election.cc:288] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:07:01.362551 16922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:07:01.362726 16918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:07:01.362732 16922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [term 2 LEADER]: Becoming Leader. State: Replica: 267036c7d28a43e298a39008ed1d73c9, State: Running, Role: LEADER
I20260923 15:07:01.362948 16922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } }
I20260923 15:07:01.363217 16922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 267036c7d28a43e298a39008ed1d73c9. Latest consensus state: current_term: 2 leader_uuid: "267036c7d28a43e298a39008ed1d73c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } } }
I20260923 15:07:01.363324 16922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:01.363219 16923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "267036c7d28a43e298a39008ed1d73c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "267036c7d28a43e298a39008ed1d73c9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41533 } } }
I20260923 15:07:01.363577 16923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:01.363963 16929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:07:01.364454 16929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:07:01.364580 16929 catalog_manager.cc:1259] Loaded cluster ID: 7a69470caf77484bba6f5a60cf688826
I20260923 15:07:01.364625 16929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:07:01.365950 16929 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:07:01.366132 16929 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 267036c7d28a43e298a39008ed1d73c9: Loaded TSK: 0
I20260923 15:07:01.366398 16929 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:07:01.435359 16920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:01.435597 16920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:01.435648 16920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:01.436888 16920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:01.436956 16920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:01.437001 16920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:07:01.438493 16920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.65.keytab
--principal=oryx/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:36181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=32913
--tserver_master_addrs=127.7.250.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:01.438764 16920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:01.439008 16920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:01.444350 16944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.444456 16920 server_base.cc:1034] running on GCE node
W20260923 15:07:01.444341 16947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:01.444667 16945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.444932 16920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:01.445232 16920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:01.446396 16920 hybrid_clock.cc:648] HybridClock initialized: now 1790176021446358 us; error 47 us; skew 500 ppm
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176021, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:01.449266 16920 init.cc:377] Logged in from keytab as oryx/127.7.250.65@KRBTEST.COM (short username oryx)
I20260923 15:07:01.449821 16920 webserver.cc:460] Webserver started at http://127.7.250.65:32913/ using document root <none> and password file <none>
I20260923 15:07:01.450098 16920 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:01.450160 16920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:01.451756 16920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:01.452432 16954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.452593 16920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:01.452695 16920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "962ba3c027504b75b5c09b88eacdc73d"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "d16fcb7cc73aed176563ad3857228490"
server_key_iv: "84f2a0b215fbeebe3ab939b05a757f1a"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.453088 16920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:01.471756 16920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:01.474159 16920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:01.474365 16920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:01.474656 16920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:01.475088 16920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:01.475128 16920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.475159 16920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:01.475183 16920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.483314 16920 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:36181
I20260923 15:07:01.483388 17067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:36181 every 8 connection(s)
I20260923 15:07:01.483783 16920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:07:01.484118 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 16920
I20260923 15:07:01.484426 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:35025
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:41533
--webserver_port=42949
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:07:01.496223 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.485032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:58935 (local address 127.7.250.126:41533)
0923 15:07:01.486033 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.486045 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.486099 (+ 54us) server_negotiation.cc:407] Connection header received
0923 15:07:01.486280 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.486288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.486359 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.486509 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.487947 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.489342 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.491184 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.491800 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.491925 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.496059 (+ 4134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:58935: BlockingRecv error: recv got EOF from 127.7.250.65:58935 (error 108)
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":85,"threads_started":1}
I20260923 15:07:01.496224 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.484657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:58935)
0923 15:07:01.484842 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.484890 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.485497 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.486583 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.486587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.487504 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.487806 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.487813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.489467 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.489469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.491082 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.491086 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.491907 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.491909 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.492082 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.492723 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.492736 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.495941 (+ 3205us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
W20260923 15:07:01.496485 17068 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:41533 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.250.126:41533: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:01.501587 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.496744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:52313)
0923 15:07:01.496798 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.496811 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.496853 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.497013 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.497014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.497133 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.497189 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.497191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.498190 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.498190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.498751 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.498752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.499246 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.499247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.499295 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.499736 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.499741 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.501536 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:07:01.501718 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.496831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:52313 (local address 127.7.250.126:41533)
0923 15:07:01.496890 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.496892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.496901 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:01.496928 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.496929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.496942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.497012 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.497259 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.498109 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.498824 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.499177 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.499250 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.501652 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:52313: BlockingRecv error: recv got EOF from 127.7.250.65:52313 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:01.507361 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.501848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:47845)
0923 15:07:01.501995 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.502008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.502060 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.502207 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.502208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.502441 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.502506 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.502508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.503184 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.503185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.504213 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.504216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.504611 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.504613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.504685 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.505112 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.505118 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.507247 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
W20260923 15:07:01.507393 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.502034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:47845 (local address 127.7.250.126:41533)
0923 15:07:01.502071 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.502073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.502086 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:07:01.502112 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.502113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.502125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.502181 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.502599 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.503047 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.504337 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.504514 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.504570 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.507324 (+ 2754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:47845: BlockingRecv error: recv got EOF from 127.7.250.65:47845 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260923 15:07:01.507582 17068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:07:01.577631 17072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:01.577872 17072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:01.577903 17072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:01.579171 17072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:01.579218 17072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:01.579245 17072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:07:01.580771 17072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.66.keytab
--principal=oryx/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:35025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=42949
--tserver_master_addrs=127.7.250.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:01.581029 17072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:01.581262 17072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:01.586643 17079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.586779 17072 server_base.cc:1034] running on GCE node
W20260923 15:07:01.586834 17078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:01.586643 17081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.587101 17072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:01.587378 17072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:01.588541 17072 hybrid_clock.cc:648] HybridClock initialized: now 1790176021588493 us; error 63 us; skew 500 ppm
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176021, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:01.591975 17072 init.cc:377] Logged in from keytab as oryx/127.7.250.66@KRBTEST.COM (short username oryx)
I20260923 15:07:01.592506 17072 webserver.cc:460] Webserver started at http://127.7.250.66:42949/ using document root <none> and password file <none>
I20260923 15:07:01.592748 17072 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:01.592809 17072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:01.594404 17072 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:01.595017 17088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.595221 17072 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:01.595314 17072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "89762907fd5c429fb93eb47d0dd41162"
format_stamp: "Formatted at 2026-09-23 15:07:00 on dist-test-slave-bb3x"
server_key: "774f55c71e30f8ec723b499fee948aa5"
server_key_iv: "c454a2b21dee6456106e022607213bf3"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.595707 17072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:01.611364 17072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:01.613942 17072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:01.614217 17072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:01.614555 17072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:01.614897 17072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:01.614950 17072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.614985 17072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:01.615012 17072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.620986 17072 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:35025
I20260923 15:07:01.621029 17202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:35025 every 8 connection(s)
I20260923 15:07:01.621387 17072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:07:01.629530 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.622220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:32859 (local address 127.7.250.126:41533)
0923 15:07:01.622303 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.622306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.622589 (+ 283us) server_negotiation.cc:407] Connection header received
0923 15:07:01.623159 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.623160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.623179 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.623239 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.624310 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.624855 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.625956 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.626185 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.626285 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.629464 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:32859: BlockingRecv error: recv got EOF from 127.7.250.66:32859 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 15:07:01.629618 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.622201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:32859)
0923 15:07:01.622407 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.622449 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.623043 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.623320 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.623324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.623969 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.624182 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.624187 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.624953 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.624954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.625855 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.625857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.626302 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.626303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.626443 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.627038 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.627047 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.629376 (+ 2329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
W20260923 15:07:01.629927 17203 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:41533 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.250.126:41533: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
I20260923 15:07:01.630647 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17072
I20260923 15:07:01.631001 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:38779
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:41533
--webserver_port=40691
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:01.641371 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.630175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:40163)
0923 15:07:01.630246 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.630293 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.630349 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.630511 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.630512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.630675 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.630740 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.630743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.631625 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.631626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.632450 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.632453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.632823 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.632824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.632891 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.639092 (+ 6201us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.639099 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.641308 (+ 2209us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260923 15:07:01.641556 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.630298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:40163 (local address 127.7.250.126:41533)
0923 15:07:01.630368 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.630370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.630380 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:01.630413 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.630414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.630424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.630487 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.630873 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.631524 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.632549 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.632747 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.632814 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.641466 (+ 8652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:40163: BlockingRecv error: recv got EOF from 127.7.250.66:40163 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:01.646921 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.641891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:50335)
0923 15:07:01.641950 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.641964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.642023 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.642168 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.642168 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.642354 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.642400 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.642402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.643342 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.643343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.643929 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.643931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.644394 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.644394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.644447 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.644981 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.644987 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.646864 (+ 1877us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 15:07:01.647085 17203 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:07:01.647090 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.641983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:50335 (local address 127.7.250.126:41533)
0923 15:07:01.642042 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.642044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.642054 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:01.642085 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.642086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.642098 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.642154 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.642467 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.643252 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.644003 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.644305 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.644372 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.646962 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:50335: BlockingRecv error: recv got EOF from 127.7.250.66:50335 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260923 15:07:01.716041 17207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:01.716252 17207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:01.716276 17207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:01.717473 17207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:01.717517 17207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:01.717545 17207 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:07:01.719069 17207 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/krb5kdc/oryx_127.7.250.67.keytab
--principal=oryx/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:38779
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=40691
--tserver_master_addrs=127.7.250.126:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17207
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:01.719326 17207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:01.719560 17207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:01.724604 17213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:01.724718 17212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.724725 17207 server_base.cc:1034] running on GCE node
W20260923 15:07:01.724604 17215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:01.725113 17207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:01.725337 17207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:01.726464 17207 hybrid_clock.cc:648] HybridClock initialized: now 1790176021726450 us; error 31 us; skew 500 ppm
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176021, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:01.728986 17207 init.cc:377] Logged in from keytab as oryx/127.7.250.67@KRBTEST.COM (short username oryx)
I20260923 15:07:01.729417 17207 webserver.cc:460] Webserver started at http://127.7.250.67:40691/ using document root <none> and password file <none>
I20260923 15:07:01.729728 17207 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:01.729817 17207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:01.731297 17207 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:01.731899 17222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.732203 17207 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:01.732300 17207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "3adf5e5a7f1a48d59aa6b04dfb701b59"
format_stamp: "Formatted at 2026-09-23 15:07:01 on dist-test-slave-bb3x"
server_key: "59d89bc3c3f2fd44ab64d30a8af5297d"
server_key_iv: "844276b508e280312b10fcfbd73e67e3"
server_key_version: "encryptionkey@0"
I20260923 15:07:01.732636 17207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:01.744028 17207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:01.745926 17207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:01.746090 17207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:01.746407 17207 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:01.746773 17207 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:01.746822 17207 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.746865 17207 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:01.746894 17207 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:01.752717 17207 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:38779
I20260923 15:07:01.752753 17335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:38779 every 8 connection(s)
I20260923 15:07:01.753095 17207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:07:01.756649 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17207
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:07:01.761945 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.753818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:42067 (local address 127.7.250.126:41533)
0923 15:07:01.753864 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.753866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.754087 (+ 221us) server_negotiation.cc:407] Connection header received
0923 15:07:01.754605 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.754606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.754618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.754692 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.755726 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.756271 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.757938 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.758335 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.758421 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.761898 (+ 3477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:42067: BlockingRecv error: recv got EOF from 127.7.250.67:42067 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:07:01.762071 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.753824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:42067)
0923 15:07:01.753973 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.754006 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.754496 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.754773 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.754776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.755407 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.755613 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.755617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.756379 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.756380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.757827 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.757830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.758430 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.758431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.758551 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.759222 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.759234 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.761809 (+ 2575us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":87,"threads_started":1}
W20260923 15:07:01.762387 17336 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:41533 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.250.126:41533: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
I20260923 15:07:01.765295 17341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.757212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.7.250.126:41533)
0923 15:07:01.757325 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.757327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.757334 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:07:01.757375 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.757376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.757385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.757441 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.758211 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.758936 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.760045 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.760263 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.760341 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.761585 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:01.761590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:01.761591 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:01.761608 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:01.763874 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:01.764261 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:01.764265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:01.764265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:01.764369 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:01.764561 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:01.764562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:01.764562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:01.764800 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:01.764963 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:01.765068 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 15:07:01.765181 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:01.767627 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.762709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:41241)
0923 15:07:01.762771 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.762780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.762835 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.762978 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.762979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.763125 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.763183 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.763185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.764027 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.764028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.764596 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.764598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.765016 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.765017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.765102 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.765638 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.765644 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.767534 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 15:07:01.767684 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.762783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:41241 (local address 127.7.250.126:41533)
0923 15:07:01.762829 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.762831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.762840 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:01.762903 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.762904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.762916 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.762972 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.763257 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.763945 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.764701 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.764922 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.765015 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.767609 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:41241: BlockingRecv error: recv got EOF from 127.7.250.67:41241 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:01 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:01.772106 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:01.767944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:52251)
0923 15:07:01.767991 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:01.768002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:01.768043 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:01.768173 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:01.768174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:01.768306 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:01.768334 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.768336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.768999 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.769000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.769543 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:01.769545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.769939 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.769940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:01.769998 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.770463 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:01.770467 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:01.772013 (+ 1546us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:07:01.772138 17071 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:01.768015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:52251 (local address 127.7.250.126:41533)
0923 15:07:01.768061 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:01.768063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:01.768072 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:01.768097 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:01.768098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:01.768108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:01.768169 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:01.768417 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.768919 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.769635 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:01.769852 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:01.769937 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:01.772080 (+ 2143us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:52251: BlockingRecv error: recv got EOF from 127.7.250.67:52251 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:07:01.772239 17336 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 15:07:02 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:02 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:02.513087 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:02.507988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:40855)
0923 15:07:02.508184 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:02.508196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:02.508266 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:02.508429 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:02.508430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:02.508599 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:02.508655 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:02.508657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.509359 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.509360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:02.509873 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:02.509874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.510347 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.510348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:02.510402 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:02.510906 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:02.510912 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:02.513024 (+ 2112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
W20260923 15:07:02.513322 17343 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:02.508110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:40855 (local address 127.7.250.126:41533)
0923 15:07:02.508239 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:02.508242 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:02.508252 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:02.508331 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:02.508331 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:02.508350 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:02.508408 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:02.508747 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.509282 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.509951 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.510185 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.510247 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:02.513241 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:40855: BlockingRecv error: recv got EOF from 127.7.250.65:40855 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
Sep 23 15:07:02 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:02 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:02.652287 17344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:02.647504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:56641)
0923 15:07:02.647660 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:02.647670 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:02.647730 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:02.647864 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:02.647865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:02.648024 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:02.648073 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:02.648074 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.648874 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.648876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:02.649490 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:02.649493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.649868 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.649868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:02.649931 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:02.650472 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:02.650477 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:02.652217 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
W20260923 15:07:02.652416 17343 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:02.647613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:56641 (local address 127.7.250.126:41533)
0923 15:07:02.647680 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:02.647681 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:02.647719 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:07:02.647779 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:02.647779 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:02.647790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:02.647852 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:02.648183 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.648783 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.649601 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.649796 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.649869 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:02.652315 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:56641: BlockingRecv error: recv got EOF from 127.7.250.66:56641 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 23 15:07:02 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:02 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:02.777654 17345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:02.772744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:40229)
0923 15:07:02.772871 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:02.772881 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:02.772932 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:02.773168 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:02.773169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:02.773329 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:02.773381 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:02.773383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.774033 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.774035 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:02.774670 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:02.774673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.775045 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.775047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:02.775114 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:02.775574 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:02.775579 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:02.777538 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
W20260923 15:07:02.777693 17343 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:02.772805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:40229 (local address 127.7.250.126:41533)
0923 15:07:02.772982 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:02.772984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:02.772993 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:02.773022 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:02.773039 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:02.773050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:02.773159 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:02.773456 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.773944 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.774748 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:02.774946 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:02.775015 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:02.777638 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:40229: BlockingRecv error: recv got EOF from 127.7.250.67:40229 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Sep 23 15:07:03 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:03 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:03.518568 17346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:03.513611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:57345)
0923 15:07:03.513802 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:03.513814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:03.513883 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:03.514045 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:03.514046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:03.514220 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:03.514296 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:03.514298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.515146 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.515147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:03.515671 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:03.515673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.516041 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.516041 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:03.516140 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:03.516599 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:03.516604 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:03.518473 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
W20260923 15:07:03.518656 17347 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:03.513694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:57345 (local address 127.7.250.126:41533)
0923 15:07:03.513806 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:03.513808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:03.513884 (+ 76us) server_negotiation.cc:407] Connection header received
0923 15:07:03.513931 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:03.513931 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:03.513943 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:03.514006 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:03.514420 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.514940 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.515767 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.515970 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.516034 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:03.518556 (+ 2522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:57345: BlockingRecv error: recv got EOF from 127.7.250.65:57345 (error 108)
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":49,"threads_started":1}
Sep 23 15:07:03 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:03 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:03.657783 17348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:03.652930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:49945)
0923 15:07:03.653172 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:03.653183 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:03.653274 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:03.653437 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:03.653437 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:03.653618 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:03.653672 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:03.653674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.654393 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.654395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:03.654925 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:03.654926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.655316 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.655317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:03.655376 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:03.655852 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:03.655858 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:03.657681 (+ 1823us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":98,"threads_started":1}
W20260923 15:07:03.657827 17347 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:03.653036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:49945 (local address 127.7.250.126:41533)
0923 15:07:03.653161 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:03.653163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:03.653255 (+ 92us) server_negotiation.cc:407] Connection header received
0923 15:07:03.653354 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:03.653354 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:03.653371 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:03.653431 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:03.653753 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.654314 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.655042 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.655233 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.655315 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:03.657761 (+ 2446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:49945: BlockingRecv error: recv got EOF from 127.7.250.66:49945 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 23 15:07:03 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:03 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:03.783399 17349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:03.778261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:38259)
0923 15:07:03.778444 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:03.778473 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:03.778541 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:03.778747 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:03.778748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:03.778933 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:03.778980 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:03.778982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.779725 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.779726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:03.780317 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:03.780318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.780701 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.780702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:03.780764 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:03.781303 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:03.781309 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:03.783295 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":101,"threads_started":1}
W20260923 15:07:03.783453 17347 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:03.778433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:38259 (local address 127.7.250.126:41533)
0923 15:07:03.778513 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:03.778515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:03.778526 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:07:03.778639 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:03.778640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:03.778655 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:03.778723 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:03.779078 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.779633 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.780410 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:03.780625 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:03.780703 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:03.783380 (+ 2677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:38259: BlockingRecv error: recv got EOF from 127.7.250.67:38259 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 15:07:04 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:04 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:04.524170 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:04.519275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:50873)
0923 15:07:04.519504 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:04.519517 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:04.519601 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:04.519799 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:04.519800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:04.519964 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:04.520036 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:04.520037 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.520760 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.520761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:04.521337 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:04.521339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.521717 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.521718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:04.521760 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:04.522222 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:04.522226 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:04.524112 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
W20260923 15:07:04.524300 17351 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:04.519402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:50873 (local address 127.7.250.126:41533)
0923 15:07:04.519554 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:04.519556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:04.519582 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:07:04.519701 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:04.519702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:04.519712 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:04.519763 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:04.520185 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.520685 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.521432 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.521639 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.521702 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:04.524210 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:50873: BlockingRecv error: recv got EOF from 127.7.250.65:50873 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
Sep 23 15:07:04 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:04 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:04.663882 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:04.658491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:59471)
0923 15:07:04.658672 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:04.658683 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:04.658745 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:04.659020 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:04.659021 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:04.659258 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:04.659308 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:04.659310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.660101 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.660103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:04.660690 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:04.660692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.661087 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.661088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:04.661152 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:04.661645 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:04.661650 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:04.663735 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
W20260923 15:07:04.664014 17351 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:04.658608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:59471 (local address 127.7.250.126:41533)
0923 15:07:04.658834 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:04.658836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:04.658844 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:07:04.658871 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:04.658872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:04.658882 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:04.658953 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:04.659404 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.660009 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.660778 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.660994 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.661084 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:04.663918 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:59471: BlockingRecv error: recv got EOF from 127.7.250.66:59471 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
Sep 23 15:07:04 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:04 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:04.788998 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:04.783995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:57479)
0923 15:07:04.784134 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:04.784146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:04.784216 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:04.784352 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:04.784353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:04.784527 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:04.784579 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:04.784581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.785228 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.785229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:04.785800 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:04.785802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.786311 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.786312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:04.786374 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:04.786868 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:04.786873 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:04.788925 (+ 2052us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":59,"threads_started":1}
W20260923 15:07:04.789191 17351 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:04.784100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:57479 (local address 127.7.250.126:41533)
0923 15:07:04.784183 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:04.784185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:04.784198 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:07:04.784266 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:04.784267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:04.784279 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:04.784332 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:04.784648 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.785145 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.785905 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:04.786172 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:04.786305 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:04.789092 (+ 2787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:57479: BlockingRecv error: recv got EOF from 127.7.250.67:57479 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 15:07:05 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:05 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:05.529634 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:05.524671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:49265)
0923 15:07:05.524834 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:05.524846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:05.524928 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:05.525085 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:05.525086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:05.525238 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:05.525296 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:05.525298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.525947 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.525947 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:05.526573 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:05.526576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.527034 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.527045 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:05.527104 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:05.527598 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:05.527604 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:05.529527 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
W20260923 15:07:05.529706 17355 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:05.524768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:49265 (local address 127.7.250.126:41533)
0923 15:07:05.524904 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:05.524906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:05.524915 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:05.524990 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:05.524990 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:05.525000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:05.525061 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:05.525355 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.525884 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.526681 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.526874 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.526932 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:05.529633 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:49265: BlockingRecv error: recv got EOF from 127.7.250.65:49265 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
Sep 23 15:07:05 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:05 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:05.669284 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:05.664490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:38627)
0923 15:07:05.664659 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:05.664670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:05.664768 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:05.664936 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:05.664937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:05.665171 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:05.665224 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:05.665226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.665897 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.665898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:05.666582 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:05.666585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.666914 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.666915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:05.666957 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:05.667434 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:05.667438 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:05.669222 (+ 1784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
W20260923 15:07:05.669350 17355 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:05.664552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:38627 (local address 127.7.250.126:41533)
0923 15:07:05.664631 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:05.664634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:05.664749 (+ 115us) server_negotiation.cc:407] Connection header received
0923 15:07:05.664838 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:05.664838 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:05.664853 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:05.664908 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:05.665300 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.665812 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.666665 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.666849 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.666905 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:05.669304 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:38627: BlockingRecv error: recv got EOF from 127.7.250.66:38627 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 23 15:07:05 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:05 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:05.794620 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:05.789475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:39577)
0923 15:07:05.789664 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:05.789680 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:05.789798 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:05.790024 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:05.790025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:05.790233 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:05.790322 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:05.790324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.791008 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.791009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:05.791618 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:05.791621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.792016 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.792017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:05.792083 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:05.792537 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:05.792542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:05.794559 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
W20260923 15:07:05.794757 17355 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:05.789597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:39577 (local address 127.7.250.126:41533)
0923 15:07:05.789689 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:05.789691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:05.789774 (+ 83us) server_negotiation.cc:407] Connection header received
0923 15:07:05.789892 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:05.789893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:05.789910 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:05.790013 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:05.790415 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.790916 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.791714 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:05.791923 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:05.792015 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:05.794666 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:39577: BlockingRecv error: recv got EOF from 127.7.250.67:39577 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 23 15:07:06 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:06 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:06.534781 17358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:06.530203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:60121)
0923 15:07:06.530371 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:06.530381 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:06.530430 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:06.530591 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:06.530592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:06.530753 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:06.530801 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:06.530803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.531497 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.531498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:06.532061 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:06.532064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.532411 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.532412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:06.532454 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:06.532962 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:06.532967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:06.534705 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":110,"threads_started":1}
W20260923 15:07:06.534929 17359 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:06.530317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:60121 (local address 127.7.250.126:41533)
0923 15:07:06.530468 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:06.530470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:06.530478 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:07:06.530501 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:06.530501 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:06.530510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:06.530579 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:06.530891 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.531424 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.532145 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.532346 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.532417 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:06.534838 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:60121: BlockingRecv error: recv got EOF from 127.7.250.65:60121 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
Sep 23 15:07:06 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:06 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:06.680019 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:06.669864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:36659)
0923 15:07:06.669997 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:06.670015 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:06.670094 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:06.670359 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:06.670360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:06.670531 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:06.670606 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:06.670607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.671450 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.671452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:06.672001 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:06.672002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.672403 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.672404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:06.672452 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:06.677866 (+ 5414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:06.677871 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:06.679950 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260923 15:07:06.680258 17359 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:06.670112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:36659 (local address 127.7.250.126:41533)
0923 15:07:06.670197 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:06.670199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:06.670210 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:07:06.670241 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:06.670243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:06.670272 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:06.670337 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:06.670785 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.671371 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.672101 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.672321 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.672392 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:06.680206 (+ 7814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:36659: BlockingRecv error: recv got EOF from 127.7.250.66:36659 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 15:07:06 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:06 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:06.800271 17361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:06.795163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:59047)
0923 15:07:06.795294 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:06.795306 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:06.795370 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:06.795513 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:06.795514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:06.795672 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:06.795721 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:06.795722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.796410 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.796411 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:06.796967 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:06.796970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.797372 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.797373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:06.797419 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:06.797924 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:06.797929 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:06.800206 (+ 2277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
W20260923 15:07:06.800457 17359 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:06.795232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:59047 (local address 127.7.250.126:41533)
0923 15:07:06.795303 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:06.795306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:06.795369 (+ 63us) server_negotiation.cc:407] Connection header received
0923 15:07:06.795415 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:06.795416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:06.795428 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:06.795500 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:06.795780 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.796331 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.797071 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:06.797293 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:06.797363 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:06.800355 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:59047: BlockingRecv error: recv got EOF from 127.7.250.67:59047 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 23 15:07:07 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:07 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:07.540163 17362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:07.535364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:47035)
0923 15:07:07.535536 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:07.535551 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:07.535617 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:07.535781 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:07.535782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:07.535956 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:07.536063 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:07.536065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.536760 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.536761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:07.537384 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:07.537386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.537698 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.537699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:07.537742 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:07.538317 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:07.538322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:07.540104 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
W20260923 15:07:07.540330 17363 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:07.535423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:47035 (local address 127.7.250.126:41533)
0923 15:07:07.535579 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:07.535582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:07.535595 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:07:07.535697 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:07.535697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:07.535713 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:07.535769 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:07.536134 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.536690 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.537453 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.537636 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.537686 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:07.540200 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:47035: BlockingRecv error: recv got EOF from 127.7.250.65:47035 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":48,"threads_started":1}
Sep 23 15:07:07 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:07 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:07.685408 17364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:07.680521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:42053)
0923 15:07:07.680824 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:07.680837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:07.680910 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:07.681104 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:07.681105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:07.681279 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:07.681328 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:07.681329 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.681998 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.681999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:07.682564 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:07.682566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.682922 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.682923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:07.682982 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:07.683452 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:07.683456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:07.685342 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":85,"threads_started":1}
W20260923 15:07:07.685554 17363 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:07.680623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:42053 (local address 127.7.250.126:41533)
0923 15:07:07.680809 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:07.680812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:07.680889 (+ 77us) server_negotiation.cc:407] Connection header received
0923 15:07:07.680970 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:07.680971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:07.680983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:07.681058 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:07.681412 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.681914 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.682658 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.682851 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.682924 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:07.685461 (+ 2537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:42053: BlockingRecv error: recv got EOF from 127.7.250.66:42053 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Sep 23 15:07:07 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:07 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:07.805986 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:07.800768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:59635)
0923 15:07:07.801072 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:07.801086 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:07.801159 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:07.801345 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:07.801346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:07.801536 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:07.801583 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:07.801585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.802314 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.802315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:07.802882 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:07.802884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.803271 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.803272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:07.803331 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:07.803821 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:07.803826 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:07.805813 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":109,"threads_started":1}
W20260923 15:07:07.805976 17363 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:07.800893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:59635 (local address 127.7.250.126:41533)
0923 15:07:07.801076 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:07.801078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:07.801153 (+ 75us) server_negotiation.cc:407] Connection header received
0923 15:07:07.801256 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:07.801257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:07.801270 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:07.801339 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:07.801667 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.802197 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.802993 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:07.803179 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:07.803272 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:07.805910 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:59635: BlockingRecv error: recv got EOF from 127.7.250.67:59635 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
Sep 23 15:07:08 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:08 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:08.545616 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:08.540687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:44875)
0923 15:07:08.540868 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:08.540878 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:08.540933 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:08.541076 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:08.541076 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:08.541236 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:08.541285 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:08.541287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.541963 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.541964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:08.542563 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:08.542567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.542988 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.542989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:08.543035 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:08.543621 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:08.543626 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:08.545487 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
W20260923 15:07:08.545660 17367 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:08.540792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:44875 (local address 127.7.250.126:41533)
0923 15:07:08.540922 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:08.540924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:08.540933 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:08.540980 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:08.540981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:08.540991 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:08.541062 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:08.541351 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.541840 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.542656 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.542856 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.542921 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:08.545574 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:44875: BlockingRecv error: recv got EOF from 127.7.250.65:44875 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
Sep 23 15:07:08 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:08 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:08.691057 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:08.685928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:37375)
0923 15:07:08.686093 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:08.686110 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:08.686161 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:08.686346 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:08.686347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:08.686507 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:08.686566 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:08.686568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.687225 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.687226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:08.687783 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:08.687784 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.688151 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.688152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:08.688202 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:08.688786 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:08.688791 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:08.690995 (+ 2204us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
W20260923 15:07:08.691216 17367 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:08.686124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:37375 (local address 127.7.250.126:41533)
0923 15:07:08.686206 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:08.686208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:08.686216 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:07:08.686241 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:08.686242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:08.686268 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:08.686323 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:08.686632 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.687138 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.687868 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.688080 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.688140 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:08.691116 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:37375: BlockingRecv error: recv got EOF from 127.7.250.66:37375 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 15:07:08 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:08 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:08.811781 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:08.806527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:50379)
0923 15:07:08.806747 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:08.806758 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:08.806835 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:08.807120 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:08.807121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:08.807340 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:08.807410 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:08.807412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.808374 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.808375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:08.808922 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:08.808923 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.809253 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.809254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:08.809299 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:08.809719 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:08.809723 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:08.811692 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":109,"threads_started":1}
W20260923 15:07:08.811866 17367 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:08.806680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:50379 (local address 127.7.250.126:41533)
0923 15:07:08.806728 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:08.806730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:08.806862 (+ 132us) server_negotiation.cc:407] Connection header received
0923 15:07:08.806902 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:08.806983 (+ 81us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:08.807000 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:08.807064 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:08.807494 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.808295 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.809003 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:08.809185 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:08.809240 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:08.811790 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:50379: BlockingRecv error: recv got EOF from 127.7.250.67:50379 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Sep 23 15:07:09 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:09 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:09.551292 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:09.546368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:47443)
0923 15:07:09.546526 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:09.546541 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:09.546614 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:09.546750 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:09.546751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:09.546909 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:09.547007 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:09.547009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.547666 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.547667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:09.548221 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:09.548222 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.548614 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.548615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:09.548685 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:09.549235 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:09.549240 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:09.551183 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":74,"threads_started":1}
W20260923 15:07:09.551342 17371 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:09.546488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:47443 (local address 127.7.250.126:41533)
0923 15:07:09.546617 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:09.546619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:09.546646 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:07:09.546669 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:09.546670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:09.546680 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:09.546729 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:09.547077 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.547579 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.548304 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.548500 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.548610 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:09.551266 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:47443: BlockingRecv error: recv got EOF from 127.7.250.65:47443 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
Sep 23 15:07:09 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:09 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:09.697046 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:09.691717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:44469)
0923 15:07:09.691917 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:09.691930 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:09.692016 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:09.692172 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:09.692173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:09.692345 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:09.692396 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:09.692408 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.693389 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.693390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:09.694032 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:09.694035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.694433 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.694434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:09.694495 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:09.694959 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:09.694964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:09.696938 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":91,"threads_started":1}
W20260923 15:07:09.697142 17371 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:09.691785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:44469 (local address 127.7.250.126:41533)
0923 15:07:09.691947 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:09.691949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:09.691997 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:07:09.692085 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:09.692086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:09.692096 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:09.692138 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:09.692479 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.693275 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.694131 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.694349 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.694435 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:09.697076 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:44469: BlockingRecv error: recv got EOF from 127.7.250.66:44469 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 23 15:07:09 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:09 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:09.817430 17373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:09.812370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:47109)
0923 15:07:09.812547 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:09.812565 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:09.812627 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:09.812806 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:09.812807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:09.812985 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:09.813032 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:09.813034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.813707 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.813708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:09.814292 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:09.814295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.814678 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.814679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:09.814730 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:09.815319 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:09.815324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:09.817358 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
W20260923 15:07:09.817585 17371 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:09.812451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:47109 (local address 127.7.250.126:41533)
0923 15:07:09.812537 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:09.812539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:09.812682 (+ 143us) server_negotiation.cc:407] Connection header received
0923 15:07:09.812711 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:09.812712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:09.812725 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:09.812771 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:09.813129 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.813622 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.814396 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:09.814588 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:09.814646 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:09.817454 (+ 2808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:47109: BlockingRecv error: recv got EOF from 127.7.250.67:47109 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 23 15:07:10 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:10 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:10.557171 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:10.551752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:33229)
0923 15:07:10.551912 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:10.551924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:10.551984 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:10.552130 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:10.552131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:10.552293 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:10.552343 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:10.552344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.553064 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.553065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:10.553682 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:10.553683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.554109 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.554110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:10.554200 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:10.554897 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:10.554903 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:10.557111 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
W20260923 15:07:10.557370 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:10.551846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:33229 (local address 127.7.250.126:41533)
0923 15:07:10.551987 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:10.551989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:10.552002 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:07:10.552033 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:10.552033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:10.552047 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:10.552113 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:10.552435 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.552942 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.553765 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.553955 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.554006 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:10.557292 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:33229: BlockingRecv error: recv got EOF from 127.7.250.65:33229 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
Sep 23 15:07:10 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:10 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:10.703099 17376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:10.697719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:48819)
0923 15:07:10.697854 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:10.697865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:10.697972 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:10.698230 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:10.698231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:10.698450 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:10.698501 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:10.698503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.699259 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.699260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:10.699808 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:10.699811 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.700200 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.700201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:10.700261 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:10.700749 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:10.700754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:10.702998 (+ 2244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
W20260923 15:07:10.703150 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:10.697753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:48819 (local address 127.7.250.126:41533)
0923 15:07:10.697807 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:10.697809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:10.697954 (+ 145us) server_negotiation.cc:407] Connection header received
0923 15:07:10.698103 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:10.698104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:10.698119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:10.698223 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:10.698592 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.699171 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.699921 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.700120 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.700203 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:10.703088 (+ 2885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:48819: BlockingRecv error: recv got EOF from 127.7.250.66:48819 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 15:07:10 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:10 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:10.823465 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:10.817948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:35561)
0923 15:07:10.818177 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:10.818190 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:10.818327 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:10.818505 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:10.818505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:10.818681 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:10.818728 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:10.818730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.819411 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.819412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:10.820197 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:10.820200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.820685 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.820687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:10.820779 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:10.821455 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:10.821460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:10.823405 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":119,"threads_started":1}
W20260923 15:07:10.823670 17375 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:10.818071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:35561 (local address 127.7.250.126:41533)
0923 15:07:10.818176 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:10.818178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:10.818292 (+ 114us) server_negotiation.cc:407] Connection header received
0923 15:07:10.818399 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:10.818400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:10.818417 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:10.818475 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:10.818813 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.819315 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.820279 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:10.820585 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:10.820645 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:10.823599 (+ 2954us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:35561: BlockingRecv error: recv got EOF from 127.7.250.67:35561 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 23 15:07:11 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:11 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:11.563113 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:11.557742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:41175)
0923 15:07:11.558030 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:11.558040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:11.558096 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:11.558310 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:11.558311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:11.558488 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:11.558539 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:11.558541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.559289 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.559290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:11.559837 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:11.559838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.560288 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.560290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:11.560370 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:11.560887 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:11.560892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:11.563055 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":113,"threads_started":1}
W20260923 15:07:11.563275 17379 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:11.557892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:41175 (local address 127.7.250.126:41533)
0923 15:07:11.558081 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:11.558083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:11.558141 (+ 58us) server_negotiation.cc:407] Connection header received
0923 15:07:11.558172 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:11.558172 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:11.558184 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:11.558241 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:11.558629 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.559198 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.559944 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.560190 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.560257 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:11.563193 (+ 2936us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:41175: BlockingRecv error: recv got EOF from 127.7.250.65:41175 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":61,"threads_started":1}
Sep 23 15:07:11 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:11 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:11.713135 17380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:11.703684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:54949)
0923 15:07:11.703906 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:11.703919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:11.704026 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:11.704186 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:11.704187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:11.704378 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:11.704428 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:11.704430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.705157 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.705159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:11.705712 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:11.705714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.706106 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.706108 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:11.706172 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:11.710895 (+ 4723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:11.710901 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:11.713018 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":121,"threads_started":1}
W20260923 15:07:11.713168 17379 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:11.703814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:54949 (local address 127.7.250.126:41533)
0923 15:07:11.703903 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:11.703905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:11.704004 (+ 99us) server_negotiation.cc:407] Connection header received
0923 15:07:11.704089 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:11.704090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:11.704103 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:11.704177 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:11.704539 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.705056 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.705818 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.706019 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.706095 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:11.713101 (+ 7006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:54949: BlockingRecv error: recv got EOF from 127.7.250.66:54949 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 15:07:11 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:11 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:11.828892 17381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:11.823934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:60323)
0923 15:07:11.824166 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:11.824180 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:11.824260 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:11.824409 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:11.824409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:11.824571 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:11.824621 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:11.824622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.825265 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.825266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:11.825824 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:11.825825 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.826200 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.826201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:11.826246 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:11.826755 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:11.826760 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:11.828832 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":94,"threads_started":1}
W20260923 15:07:11.829066 17379 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:11.824137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:60323 (local address 127.7.250.126:41533)
0923 15:07:11.824202 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:11.824204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:11.824236 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:07:11.824317 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:11.824318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:11.824330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:11.824379 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:11.824681 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.825184 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.825915 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:11.826123 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:11.826180 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:11.828915 (+ 2735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:60323: BlockingRecv error: recv got EOF from 127.7.250.67:60323 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 23 15:07:12 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:12 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:12.569031 17382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:12.563997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:49261)
0923 15:07:12.564138 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:12.564166 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:12.564223 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:12.564400 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:12.564401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:12.564577 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:12.564622 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:12.564624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.565658 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.565659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:12.566217 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:12.566220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.566578 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.566579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:12.566626 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:12.567052 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:12.567057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:12.568961 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
W20260923 15:07:12.569164 17383 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:12.564045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:49261 (local address 127.7.250.126:41533)
0923 15:07:12.564173 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:12.564175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:12.564214 (+ 39us) server_negotiation.cc:407] Connection header received
0923 15:07:12.564282 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:12.564283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:12.564294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:12.564349 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:12.564719 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.565561 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.566310 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.566508 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.566565 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:12.569072 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:49261: BlockingRecv error: recv got EOF from 127.7.250.65:49261 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
Sep 23 15:07:12 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:12 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:12.719107 17384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:12.713721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:43171)
0923 15:07:12.713844 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:12.713855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:12.713919 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:12.714190 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:12.714193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:12.714470 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:12.714537 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:12.714539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.715459 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.715461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:12.716050 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:12.716052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.716451 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.716451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:12.716497 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:12.716948 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:12.716953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:12.719030 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":71,"threads_started":1}
W20260923 15:07:12.719285 17383 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:12.713850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:43171 (local address 127.7.250.126:41533)
0923 15:07:12.713936 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:12.713939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:12.713951 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:07:12.713987 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:12.713988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:12.714002 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:12.714130 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:12.714631 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.715350 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.716143 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.716352 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.716450 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:12.719171 (+ 2721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:43171: BlockingRecv error: recv got EOF from 127.7.250.66:43171 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 23 15:07:12 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:12 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:12.834563 17385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:12.829471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:55115)
0923 15:07:12.829621 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:12.829633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:12.829701 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:12.829877 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:12.829877 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:12.830044 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:12.830092 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:12.830093 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.831012 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.831013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:12.831534 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:12.831535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.831930 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.831931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:12.831971 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:12.832499 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:12.832504 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:12.834465 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
W20260923 15:07:12.834609 17383 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:12.829570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:55115 (local address 127.7.250.126:41533)
0923 15:07:12.829672 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:12.829679 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:12.829693 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:07:12.829769 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:12.829770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:12.829783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:12.829854 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:12.830199 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.830926 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.831649 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:12.831852 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:12.831916 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:12.834539 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:55115: BlockingRecv error: recv got EOF from 127.7.250.67:55115 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 23 15:07:13 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:13 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:13.574995 17386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:13.569651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:47685)
0923 15:07:13.569944 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:13.569957 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:13.570038 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:13.570295 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:13.570296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:13.570542 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:13.570621 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:13.570623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.571351 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.571352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:13.571947 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:13.571949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.572326 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.572327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:13.572380 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:13.572925 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:13.572930 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:13.574935 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":95,"threads_started":1}
W20260923 15:07:13.575201 17387 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:13.569762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:47685 (local address 127.7.250.126:41533)
0923 15:07:13.569951 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:13.569953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:13.570049 (+ 96us) server_negotiation.cc:407] Connection header received
0923 15:07:13.570105 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:13.570106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:13.570122 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:13.570224 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:13.570706 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.571254 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.572034 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.572241 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.572298 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:13.575108 (+ 2810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:47685: BlockingRecv error: recv got EOF from 127.7.250.65:47685 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":54,"threads_started":1}
Sep 23 15:07:13 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:13 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:13.724864 17388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:13.719702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:45425)
0923 15:07:13.719956 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:13.719968 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:13.720080 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:13.720247 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:13.720248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:13.720447 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:13.720494 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:13.720496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.721175 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.721176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:13.721720 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:13.721722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.722114 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.722115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:13.722171 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:13.722785 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:13.722791 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:13.724796 (+ 2005us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":183,"threads_started":1}
W20260923 15:07:13.725052 17387 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:13.719821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:45425 (local address 127.7.250.126:41533)
0923 15:07:13.719942 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:13.719944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:13.720060 (+ 116us) server_negotiation.cc:407] Connection header received
0923 15:07:13.720147 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:13.720148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:13.720163 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:13.720240 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:13.720590 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.721087 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.721841 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.722034 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.722116 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:13.724982 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:45425: BlockingRecv error: recv got EOF from 127.7.250.66:45425 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 23 15:07:13 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:13 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:13.840359 17389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:13.835138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:37763)
0923 15:07:13.835344 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:13.835357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:13.835443 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:13.835647 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:13.835648 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:13.835842 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:13.835888 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:13.835890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.836626 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.836628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:13.837161 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:13.837162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.837537 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.837538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:13.837598 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:13.838149 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:13.838153 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:13.840296 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":97,"threads_started":1}
W20260923 15:07:13.840440 17387 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:13.835258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:37763 (local address 127.7.250.126:41533)
0923 15:07:13.835333 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:13.835335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:13.835441 (+ 106us) server_negotiation.cc:407] Connection header received
0923 15:07:13.835548 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:13.835550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:13.835565 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:13.835641 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:13.836017 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.836522 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.837250 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:13.837445 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:13.837536 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:13.840392 (+ 2856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:37763: BlockingRecv error: recv got EOF from 127.7.250.67:37763 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 15:07:14 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:14 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:14.580677 17390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:14.575646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:51031)
0923 15:07:14.575799 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:14.575815 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:14.575893 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:14.576030 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:14.576031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:14.576216 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:14.576268 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:14.576269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.576981 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.576982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:14.577592 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:14.577594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.578029 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.578031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:14.578085 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:14.578612 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:14.578617 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:14.580612 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260923 15:07:14.580853 17391 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:14.575723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:51031 (local address 127.7.250.126:41533)
0923 15:07:14.575833 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:14.575835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:14.575873 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:07:14.575955 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:14.575955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:14.575966 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:14.576025 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:14.576369 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.576885 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.577688 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.577927 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.577987 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:14.580740 (+ 2753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:51031: BlockingRecv error: recv got EOF from 127.7.250.65:51031 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":63,"threads_started":1}
Sep 23 15:07:14 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:14 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:14.730728 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:14.725435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:39703)
0923 15:07:14.725584 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:14.725594 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:14.725661 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:14.725866 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:14.725867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:14.726056 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:14.726101 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:14.726102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.726849 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.726850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:14.727387 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:14.727390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.727876 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.727878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:14.727955 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:14.728489 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:14.728495 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:14.730636 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260923 15:07:14.730865 17391 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:14.725630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:39703 (local address 127.7.250.126:41533)
0923 15:07:14.725696 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:14.725698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:14.725708 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:14.725744 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:14.725744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:14.725754 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:14.725856 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:14.726191 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.726750 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.727485 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.727688 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.727842 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:14.730798 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:39703: BlockingRecv error: recv got EOF from 127.7.250.66:39703 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 23 15:07:14 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:14 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:14.845885 17393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:14.840913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:53723)
0923 15:07:14.841138 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:14.841152 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:14.841231 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:14.841477 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:14.841478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:14.841721 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:14.841783 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:14.841785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.842552 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.842553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:14.843132 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:14.843134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.843521 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.843522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:14.843572 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:14.844023 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:14.844028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:14.845818 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":113,"threads_started":1}
W20260923 15:07:14.846004 17391 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:14.841110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:53723 (local address 127.7.250.126:41533)
0923 15:07:14.841181 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:14.841183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:14.841195 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:07:14.841363 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:14.841364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:14.841381 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:14.841443 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:14.841937 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.842465 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.843223 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:14.843435 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:14.843495 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:14.845924 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:53723: BlockingRecv error: recv got EOF from 127.7.250.67:53723 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 15:07:15 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:15 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:15.586948 17394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:15.581366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:33719)
0923 15:07:15.581539 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:15.581550 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:15.581611 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:15.581867 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:15.581868 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:15.582187 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:15.582307 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:15.582310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.583034 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.583036 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:15.583565 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:15.583568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.583941 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.583942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:15.584008 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:15.584526 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:15.584531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:15.586886 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":117,"threads_started":1}
W20260923 15:07:15.587173 17395 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:15.581522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:33719 (local address 127.7.250.126:41533)
0923 15:07:15.581682 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:15.581684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:15.581693 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:15.581720 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:15.581721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:15.581753 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:15.581836 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:15.582385 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.582953 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.583654 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.583854 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.583940 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:15.587096 (+ 3156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:33719: BlockingRecv error: recv got EOF from 127.7.250.65:33719 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
Sep 23 15:07:15 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:15 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:15.736481 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:15.731291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:40795)
0923 15:07:15.731491 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:15.731502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:15.731574 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:15.731718 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:15.731719 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:15.731925 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:15.731975 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:15.731977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.732745 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.732746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:15.733308 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:15.733311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.733689 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.733690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:15.733746 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:15.734430 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:15.734435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:15.736388 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":101,"threads_started":1}
W20260923 15:07:15.736646 17395 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:15.731421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:40795 (local address 127.7.250.126:41533)
0923 15:07:15.731474 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:15.731476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:15.731555 (+ 79us) server_negotiation.cc:407] Connection header received
0923 15:07:15.731632 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:15.731633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:15.731643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:15.731700 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:15.732073 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.732649 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.733404 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.733607 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.733663 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:15.736579 (+ 2916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:40795: BlockingRecv error: recv got EOF from 127.7.250.66:40795 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 23 15:07:15 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:15 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.67@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:15.851140 17397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:15.846415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.67:57991)
0923 15:07:15.846579 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:15.846594 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:15.846666 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:15.846902 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:15.846903 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:15.847089 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:15.847134 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:15.847136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.847845 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.847846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:15.848375 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:15.848377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.848727 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.848727 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:15.848787 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:15.849237 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:15.849241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:15.851081 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
W20260923 15:07:15.851259 17395 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:15.846456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:57991 (local address 127.7.250.126:41533)
0923 15:07:15.846523 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:15.846526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:15.846652 (+ 126us) server_negotiation.cc:407] Connection header received
0923 15:07:15.846725 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:15.846726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:15.846745 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:15.846892 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:15.847215 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.847719 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.848462 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:15.848653 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:15.848728 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:15.851169 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.67:57991: BlockingRecv error: recv got EOF from 127.7.250.67:57991 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 15:07:16 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:16 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.65@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:16.592785 17398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:16.587542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.65:43963)
0923 15:07:16.587673 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:16.587686 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:16.587745 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:16.587955 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:16.587956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:16.588136 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:16.588194 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:16.588195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.589110 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.589111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:16.589636 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:16.589638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.590042 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.590043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:16.590095 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:16.590658 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:16.590662 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:16.592725 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
W20260923 15:07:16.592989 17399 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:16.587632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:43963 (local address 127.7.250.126:41533)
0923 15:07:16.587736 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:16.587738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:16.587748 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:16.587846 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:16.587846 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:16.587859 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:16.587924 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:16.588350 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.589007 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.589738 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.589942 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.590010 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:16.592839 (+ 2829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.65:43963: BlockingRecv error: recv got EOF from 127.7.250.65:43963 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":48,"threads_started":1}
Sep 23 15:07:16 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:07:16 dist-test-slave-bb3x krb5kdc[16333](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.250.66@KRBTEST.COM for oryx/127.7.250.126@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:07:16.745111 17400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:16.737169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41533 (local address 127.7.250.66:33729)
0923 15:07:16.737338 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:16.737349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:16.737410 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:16.737588 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:16.737589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:16.737747 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:16.737798 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:16.737799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.738517 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.738518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:16.739079 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:16.739081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.739443 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.739444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:16.739490 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:16.742893 (+ 3403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:16.742903 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:16.745036 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.250.126:41533: Server oryx/127.7.250.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
W20260923 15:07:16.745186 17399 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:16.737227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:33729 (local address 127.7.250.126:41533)
0923 15:07:16.737313 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:16.737315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:16.737459 (+ 144us) server_negotiation.cc:407] Connection header received
0923 15:07:16.737496 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:16.737497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:16.737512 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:16.737570 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:16.737866 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.738438 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.739166 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.739354 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.739427 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:16.745138 (+ 5711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.250.66:33729: BlockingRecv error: recv got EOF from 127.7.250.66:33729 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 15:07:16.764834 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16920
I20260923 15:07:16.769537 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17072
I20260923 15:07:16.774142 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17207
I20260923 15:07:16.778548 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 16849
2026-09-23T15:07:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17626 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T15:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:07:16Z Disabled control of system clock
I20260923 15:07:16.807777 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40897
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:41203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:40897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 15:07:16.890007 17404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:16.890231 17404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:16.890358 17404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:16.891587 17404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:07:16.891630 17404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:16.891644 17404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:16.891656 17404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:07:16.891669 17404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:07:16.893086 17404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:40897
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:16.893364 17404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:16.893692 17404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:16.899292 17412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:16.899292 17409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:16.899345 17410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:16.899580 17404 server_base.cc:1034] running on GCE node
I20260923 15:07:16.899940 17404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:16.900271 17404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:16.901477 17404 hybrid_clock.cc:648] HybridClock initialized: now 1790176036901452 us; error 43 us; skew 500 ppm
I20260923 15:07:16.902832 17404 webserver.cc:460] Webserver started at http://127.7.250.126:44959/ using document root <none> and password file <none>
I20260923 15:07:16.903090 17404 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:16.903162 17404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:16.903323 17404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:16.904448 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "3a1fe6729dd347168be759edd7a98d78"
format_stamp: "Formatted at 2026-09-23 15:07:16 on dist-test-slave-bb3x"
server_key: "6482d5dd2608bad7517040a84ae342cb"
server_key_iv: "9de83c738190e7d279fe396160edb38c"
server_key_version: "encryptionkey@0"
I20260923 15:07:16.904819 17404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "3a1fe6729dd347168be759edd7a98d78"
format_stamp: "Formatted at 2026-09-23 15:07:16 on dist-test-slave-bb3x"
server_key: "6482d5dd2608bad7517040a84ae342cb"
server_key_iv: "9de83c738190e7d279fe396160edb38c"
server_key_version: "encryptionkey@0"
I20260923 15:07:16.906177 17404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260923 15:07:16.906924 17418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:16.907101 17404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:07:16.907208 17404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "3a1fe6729dd347168be759edd7a98d78"
format_stamp: "Formatted at 2026-09-23 15:07:16 on dist-test-slave-bb3x"
server_key: "6482d5dd2608bad7517040a84ae342cb"
server_key_iv: "9de83c738190e7d279fe396160edb38c"
server_key_version: "encryptionkey@0"
I20260923 15:07:16.907290 17404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:16.922187 17404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:16.922602 17404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:16.922768 17404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:16.926669 17404 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:40897
I20260923 15:07:16.926710 17470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:40897 every 8 connection(s)
I20260923 15:07:16.927073 17404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:07:16.927722 17472 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:16.930452 17472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78: Bootstrap starting.
I20260923 15:07:16.931049 17472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:16.931358 17472 log.cc:824] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:16.932102 17472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78: No bootstrap required, opened a new log
I20260923 15:07:16.932437 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17404
I20260923 15:07:16.932524 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:07:16.932758 8169 external_mini_cluster.cc:1428] Setting key 4ea8fff70c2290fd7b5a6a8260c968e1
I20260923 15:07:16.934410 17472 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fe6729dd347168be759edd7a98d78" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40897 } }
I20260923 15:07:16.934588 17472 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:16.934625 17472 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1fe6729dd347168be759edd7a98d78, State: Initialized, Role: FOLLOWER
I20260923 15:07:16.934792 17472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fe6729dd347168be759edd7a98d78" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40897 } }
I20260923 15:07:16.934887 17472 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:07:16.934921 17472 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:07:16.934974 17472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:16.935726 17472 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fe6729dd347168be759edd7a98d78" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40897 } }
I20260923 15:07:16.935837 17472 leader_election.cc:302] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3a1fe6729dd347168be759edd7a98d78; no voters:
I20260923 15:07:16.935985 17472 leader_election.cc:288] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:07:16.936085 17477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:07:16.936261 17477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1fe6729dd347168be759edd7a98d78, State: Running, Role: LEADER
I20260923 15:07:16.936319 17472 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:07:16.937889 17477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fe6729dd347168be759edd7a98d78" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40897 } }
I20260923 15:07:16.938519 17478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3a1fe6729dd347168be759edd7a98d78. Latest consensus state: current_term: 1 leader_uuid: "3a1fe6729dd347168be759edd7a98d78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fe6729dd347168be759edd7a98d78" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40897 } } }
I20260923 15:07:16.938558 17479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3a1fe6729dd347168be759edd7a98d78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3a1fe6729dd347168be759edd7a98d78" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40897 } } }
W20260923 15:07:16.939369 17491 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260923 15:07:16.939419 17479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:16.939388 17478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78 [sys.catalog]: This master's current role is: LEADER
W20260923 15:07:16.939440 17491 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:07:16.939700 17494 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:07:16.939845 17494 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:07:16.940433 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:16.933377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.7.250.126:40897)
0923 15:07:16.933617 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:16.933626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:16.933661 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:07:16.934355 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:16.934364 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:16.934463 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:16.934618 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:16.934860 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.936642 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.937648 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:16.938369 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:16.938510 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:16.939552 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:16.939556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:16.939557 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:16.939811 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:16.939862 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:16.939875 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:16.939966 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:16.940108 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 15:07:16.940202 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260923 15:07:16.941957 17494 catalog_manager.cc:1347] Generated new cluster ID: 304d4ee02f1f47b39ccde165d2b7f18b
I20260923 15:07:16.942005 17494 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:07:16.947934 17494 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:07:16.948487 17494 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:07:16.965173 17494 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3a1fe6729dd347168be759edd7a98d78: Generated new TSK 0
I20260923 15:07:16.965442 17494 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:07:16.975320 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40897
--builtin_ntp_servers=127.7.250.84:41203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 15:07:17.079931 17496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.080149 17496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.080171 17496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.081439 17496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.081488 17496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.081514 17496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:07:17.082906 17496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.083179 17496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.083518 17496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.089493 17504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.089504 17501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.089504 17502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.089656 17496 server_base.cc:1034] running on GCE node
I20260923 15:07:17.090929 17496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.091221 17496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.092432 17496 hybrid_clock.cc:648] HybridClock initialized: now 1790176037092407 us; error 50 us; skew 500 ppm
I20260923 15:07:17.094027 17496 webserver.cc:460] Webserver started at http://127.7.250.65:43147/ using document root <none> and password file <none>
I20260923 15:07:17.094377 17496 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.094468 17496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.094614 17496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.096112 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "a4ceb35cb1fa4d89ad70f8c6bc4b8213"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "fa44bbdce16617fa145fa8aabc9c1098"
server_key_iv: "13d70ab739d2d278de9e9ebb307d50b9"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.096576 17496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "a4ceb35cb1fa4d89ad70f8c6bc4b8213"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "fa44bbdce16617fa145fa8aabc9c1098"
server_key_iv: "13d70ab739d2d278de9e9ebb307d50b9"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.098315 17496 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260923 15:07:17.099150 17510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.099316 17496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:17.099402 17496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "a4ceb35cb1fa4d89ad70f8c6bc4b8213"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "fa44bbdce16617fa145fa8aabc9c1098"
server_key_iv: "13d70ab739d2d278de9e9ebb307d50b9"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.099505 17496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.112553 17496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.112845 17496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.112953 17496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.113166 17496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:17.113523 17496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:17.113583 17496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.113626 17496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:17.113651 17496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.119298 17496 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:46165
I20260923 15:07:17.119413 17623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:46165 every 8 connection(s)
I20260923 15:07:17.119758 17496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:07:17.121184 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17496
I20260923 15:07:17.121294 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:07:17.121538 8169 external_mini_cluster.cc:1428] Setting key d06e91f6cb4c3dd03e75828096b63ab2
I20260923 15:07:17.122643 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40897
--builtin_ntp_servers=127.7.250.84:41203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 15:07:17.126607 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.120530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:57899 (local address 127.7.250.126:40897)
0923 15:07:17.120607 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.120609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.120821 (+ 212us) server_negotiation.cc:407] Connection header received
0923 15:07:17.121662 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.121664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.121675 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.121758 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.122689 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.123254 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.124557 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.124888 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.124966 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:17.126194 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.126195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.126196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.126203 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.126219 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.126225 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.126366 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.126490 (+ 124us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.126530 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:07:17.126889 17626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.120493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40897 (local address 127.7.250.65:57899)
0923 15:07:17.120692 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:17.120726 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:17.121532 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:17.121871 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:17.121876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:17.121887 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:17.122591 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:17.122599 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.123352 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.123353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:17.124439 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:17.124442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.125007 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.125009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:17.125184 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:17.125998 (+ 814us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:07:17.125999 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:17.126040 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:07:17.126073 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:17.126376 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:17.126384 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:17.126388 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:17.126473 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:07:17.126603 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260923 15:07:17.127497 17624 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40897
I20260923 15:07:17.127593 17624 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:17.127868 17624 heartbeater.cc:511] Master 127.7.250.126:40897 requested a full tablet report, sending...
I20260923 15:07:17.128424 17435 ts_manager.cc:194] Registered new tserver with Master: a4ceb35cb1fa4d89ad70f8c6bc4b8213 (127.7.250.65:46165)
I20260923 15:07:17.129079 17435 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.65:57899
W20260923 15:07:17.204878 17627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.205109 17627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.205147 17627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.206471 17627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.206527 17627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.206570 17627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:07:17.207944 17627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.208207 17627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.208503 17627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.213717 17633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.213701 17632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.213856 17627 server_base.cc:1034] running on GCE node
W20260923 15:07:17.213727 17635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.214226 17627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.214514 17627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.215659 17627 hybrid_clock.cc:648] HybridClock initialized: now 1790176037215646 us; error 27 us; skew 500 ppm
I20260923 15:07:17.217038 17627 webserver.cc:460] Webserver started at http://127.7.250.66:34345/ using document root <none> and password file <none>
I20260923 15:07:17.217295 17627 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.217346 17627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.217520 17627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.218710 17627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "ea3eb46dd1a34ca9af57f6c6332b2ae4"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "254a23e0eaf7bd40c37cb0743fba978a"
server_key_iv: "bf49f52282a5a3526c33141dd2f2ea7e"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.219063 17627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "ea3eb46dd1a34ca9af57f6c6332b2ae4"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "254a23e0eaf7bd40c37cb0743fba978a"
server_key_iv: "bf49f52282a5a3526c33141dd2f2ea7e"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.220435 17627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:17.221230 17641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.221444 17627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:17.221534 17627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "ea3eb46dd1a34ca9af57f6c6332b2ae4"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "254a23e0eaf7bd40c37cb0743fba978a"
server_key_iv: "bf49f52282a5a3526c33141dd2f2ea7e"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.221606 17627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.237018 17627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.237367 17627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.237520 17627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.237766 17627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:17.238173 17627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:17.238210 17627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.238234 17627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:17.238246 17627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.244258 17627 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:35487
I20260923 15:07:17.244300 17756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:35487 every 8 connection(s)
I20260923 15:07:17.244652 17627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:07:17.248373 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17627
I20260923 15:07:17.248481 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:07:17.248674 8169 external_mini_cluster.cc:1428] Setting key 0f6009cac0dd976ae9569a5e1590bda0
I20260923 15:07:17.249606 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40897
--builtin_ntp_servers=127.7.250.84:41203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 15:07:17.250357 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.245481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:41071 (local address 127.7.250.126:40897)
0923 15:07:17.245573 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.245576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.245788 (+ 212us) server_negotiation.cc:407] Connection header received
0923 15:07:17.246397 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.246399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.246416 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.246475 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.247171 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.247672 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.248835 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.249036 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.249113 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:17.250014 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.250016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.250016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.250021 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.250036 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.250042 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.250107 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.250191 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.250224 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:07:17.250607 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.245466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40897 (local address 127.7.250.66:41071)
0923 15:07:17.245667 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:17.245701 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:17.246236 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:17.246550 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:17.246555 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:17.246566 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:17.247076 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:17.247080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.247796 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.247797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:17.248759 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:17.248762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.249132 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.249132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:17.249276 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:17.249854 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:07:17.249854 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:17.249883 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:07:17.249906 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:17.250109 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:17.250115 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:17.250119 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:17.250192 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:07:17.250320 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260923 15:07:17.251024 17757 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40897
I20260923 15:07:17.251138 17757 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:17.251322 17757 heartbeater.cc:511] Master 127.7.250.126:40897 requested a full tablet report, sending...
I20260923 15:07:17.251688 17435 ts_manager.cc:194] Registered new tserver with Master: ea3eb46dd1a34ca9af57f6c6332b2ae4 (127.7.250.66:35487)
I20260923 15:07:17.252079 17435 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.66:41071
W20260923 15:07:17.332093 17760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.332341 17760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.332378 17760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.333571 17760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.333642 17760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.333693 17760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:07:17.335230 17760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.335501 17760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.335855 17760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.341447 17765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.341442 17766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.341439 17768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.341516 17760 server_base.cc:1034] running on GCE node
I20260923 15:07:17.342018 17760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.342361 17760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.343549 17760 hybrid_clock.cc:648] HybridClock initialized: now 1790176037343537 us; error 39 us; skew 500 ppm
I20260923 15:07:17.345144 17760 webserver.cc:460] Webserver started at http://127.7.250.67:41987/ using document root <none> and password file <none>
I20260923 15:07:17.345432 17760 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.345526 17760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.345690 17760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.346951 17760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "ae6d3989ace3482a98965e3a2215b70a"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "de613ada1206802ab4d092eb0c3e986b"
server_key_iv: "812596f01b359dda1e8b60a7328dffb3"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.347347 17760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "ae6d3989ace3482a98965e3a2215b70a"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "de613ada1206802ab4d092eb0c3e986b"
server_key_iv: "812596f01b359dda1e8b60a7328dffb3"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.348843 17760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:17.349610 17774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.349787 17760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:07:17.349886 17760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "ae6d3989ace3482a98965e3a2215b70a"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "de613ada1206802ab4d092eb0c3e986b"
server_key_iv: "812596f01b359dda1e8b60a7328dffb3"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.349978 17760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.360738 17760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.361102 17760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.361275 17760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.361547 17760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:17.361927 17760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:17.361985 17760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.362031 17760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:17.362064 17760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.369114 17760 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:40703
I20260923 15:07:17.369163 17890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:40703 every 8 connection(s)
I20260923 15:07:17.369549 17760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:07:17.374889 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17760
I20260923 15:07:17.375018 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:07:17.375283 17476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.370467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:53363 (local address 127.7.250.126:40897)
0923 15:07:17.370535 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.370537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.370789 (+ 252us) server_negotiation.cc:407] Connection header received
0923 15:07:17.371417 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.371419 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.371433 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.371490 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.372258 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.372763 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.373758 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.373991 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.374072 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:17.374981 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.374983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.374983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.374988 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.375003 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.375008 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.375061 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.375195 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.375220 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:07:17.375309 8169 external_mini_cluster.cc:1428] Setting key f44b10f0382caa009efab8c12614b241
I20260923 15:07:17.375600 17893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.370469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40897 (local address 127.7.250.67:53363)
0923 15:07:17.370659 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:17.370694 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:17.371268 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:17.371572 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:17.371575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:17.371584 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:17.372156 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:17.372161 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.372864 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.372872 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:17.373663 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:17.373665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:17.374082 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:17.374083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:17.374222 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:17.374826 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:07:17.374827 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:17.374856 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:07:17.374886 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:17.375074 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:17.375082 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:17.375086 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:17.375199 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:07:17.375330 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260923 15:07:17.376155 17891 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40897
I20260923 15:07:17.376260 17891 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:17.376451 17891 heartbeater.cc:511] Master 127.7.250.126:40897 requested a full tablet report, sending...
I20260923 15:07:17.376859 17435 ts_manager.cc:194] Registered new tserver with Master: ae6d3989ace3482a98965e3a2215b70a (127.7.250.67:40703)
I20260923 15:07:17.377247 17435 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.67:53363
I20260923 15:07:17.387914 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 15:07:17.390125 17476 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:17.389381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.7.250.126:40897)
0923 15:07:17.389462 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.389465 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.389633 (+ 168us) server_negotiation.cc:407] Connection header received
0923 15:07:17.389700 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.389701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.389711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.389762 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.390028 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42776: BlockingRecv error: recv got EOF from 127.0.0.1:42776 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 15:07:17.390201 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17496
I20260923 15:07:17.394999 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17627
I20260923 15:07:17.399374 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17760
I20260923 15:07:17.403713 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17404
2026-09-23T15:07:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (623 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T15:07:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:07:17Z Disabled control of system clock
I20260923 15:07:17.430773 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:36399
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:36399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 15:07:17.512750 17902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.512972 17902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.512995 17902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.514098 17902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:07:17.514138 17902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.514153 17902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.514171 17902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:07:17.514183 17902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:07:17.515559 17902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:36399
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:36399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.515762 17902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.516005 17902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.521153 17907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.521158 17910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.521286 17902 server_base.cc:1034] running on GCE node
W20260923 15:07:17.521163 17908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.521638 17902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.521893 17902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.523028 17902 hybrid_clock.cc:648] HybridClock initialized: now 1790176037523008 us; error 35 us; skew 500 ppm
I20260923 15:07:17.524294 17902 webserver.cc:460] Webserver started at http://127.7.250.126:44149/ using document root <none> and password file <none>
I20260923 15:07:17.524559 17902 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.524607 17902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.524698 17902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.525772 17902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "d97ef04fee3d4edf8f1683a9d4e22b55"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "c82b7ea216391734f851908fb385a93c"
server_key_iv: "08da09f21dca1e176a7f32ed74ab6ef3"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.526074 17902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "d97ef04fee3d4edf8f1683a9d4e22b55"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "c82b7ea216391734f851908fb385a93c"
server_key_iv: "08da09f21dca1e176a7f32ed74ab6ef3"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.527397 17902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:07:17.528141 17916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.528388 17902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:17.528483 17902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "d97ef04fee3d4edf8f1683a9d4e22b55"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "c82b7ea216391734f851908fb385a93c"
server_key_iv: "08da09f21dca1e176a7f32ed74ab6ef3"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.528548 17902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.532132 17902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.532449 17902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.532613 17902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.536701 17902 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:36399
I20260923 15:07:17.536746 17969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:36399 every 8 connection(s)
I20260923 15:07:17.537147 17902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:07:17.537874 17970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:17.540736 17970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55: Bootstrap starting.
I20260923 15:07:17.541325 17970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:17.541637 17970 log.cc:824] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:17.542407 17970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55: No bootstrap required, opened a new log
I20260923 15:07:17.544039 17970 raft_consensus.cc:348] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36399 } }
I20260923 15:07:17.544175 17970 raft_consensus.cc:374] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:17.544198 17970 raft_consensus.cc:729] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97ef04fee3d4edf8f1683a9d4e22b55, State: Initialized, Role: FOLLOWER
I20260923 15:07:17.544382 17970 consensus_queue.cc:260] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36399 } }
I20260923 15:07:17.544453 17970 raft_consensus.cc:388] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:07:17.544476 17970 raft_consensus.cc:482] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:07:17.544543 17970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:17.544871 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17902
I20260923 15:07:17.544982 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:07:17.545184 8169 external_mini_cluster.cc:1428] Setting key e20154883c133d1ed27bbaa599af8316
I20260923 15:07:17.545372 17970 raft_consensus.cc:504] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36399 } }
I20260923 15:07:17.545542 17970 leader_election.cc:302] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d97ef04fee3d4edf8f1683a9d4e22b55; no voters:
I20260923 15:07:17.545773 17970 leader_election.cc:288] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:07:17.546073 17976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:07:17.546094 17970 sys_catalog.cc:564] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:07:17.546307 17976 raft_consensus.cc:686] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [term 1 LEADER]: Becoming Leader. State: Replica: d97ef04fee3d4edf8f1683a9d4e22b55, State: Running, Role: LEADER
I20260923 15:07:17.546417 17976 consensus_queue.cc:237] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36399 } }
I20260923 15:07:17.546747 17976 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36399 } } }
I20260923 15:07:17.546820 17976 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:17.546942 17976 sys_catalog.cc:455] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d97ef04fee3d4edf8f1683a9d4e22b55. Latest consensus state: current_term: 1 leader_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d97ef04fee3d4edf8f1683a9d4e22b55" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36399 } } }
I20260923 15:07:17.547001 17976 sys_catalog.cc:458] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55 [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:17.547776 17980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:07:17.548087 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.545759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.7.250.126:36399)
0923 15:07:17.546088 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.546099 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.546135 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:07:17.546350 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.546358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.546426 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.546619 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.547389 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.547393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.547393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.547397 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.547443 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.547458 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.547528 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.547812 (+ 284us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.547877 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":103,"threads_started":1}
I20260923 15:07:17.548341 17980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:07:17.550143 17980 catalog_manager.cc:1347] Generated new cluster ID: ed9c3f4ba0cd48f8b1e2ecd8ab3d7c16
I20260923 15:07:17.550203 17980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:07:17.558516 17980 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:07:17.558692 17980 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:07:17.567577 17980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d97ef04fee3d4edf8f1683a9d4e22b55: Generated new TSK 0
I20260923 15:07:17.567813 17980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:07:17.572259 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36399
--builtin_ntp_servers=127.7.250.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 15:07:17.658192 17994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.658509 17994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.658548 17994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.659746 17994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.659817 17994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.659860 17994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:07:17.661264 17994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.661567 17994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.661899 17994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.667219 18002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.667259 18000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.667230 17999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.668094 17994 server_base.cc:1034] running on GCE node
I20260923 15:07:17.668372 17994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.668681 17994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.669937 17994 hybrid_clock.cc:648] HybridClock initialized: now 1790176037669883 us; error 69 us; skew 500 ppm
I20260923 15:07:17.671489 17994 webserver.cc:460] Webserver started at http://127.7.250.65:44129/ using document root <none> and password file <none>
I20260923 15:07:17.671782 17994 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.671870 17994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.672048 17994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.673247 17994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "4c102c6e4a844a1bb630ffd8deb415be"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "e7368be92c621b12b916d0a09ee478a0"
server_key_iv: "32940e49f39b857d6bc555c50cd56848"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.673668 17994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "4c102c6e4a844a1bb630ffd8deb415be"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "e7368be92c621b12b916d0a09ee478a0"
server_key_iv: "32940e49f39b857d6bc555c50cd56848"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.675289 17994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:07:17.676174 18008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.676417 17994 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:17.676522 17994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "4c102c6e4a844a1bb630ffd8deb415be"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "e7368be92c621b12b916d0a09ee478a0"
server_key_iv: "32940e49f39b857d6bc555c50cd56848"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.676596 17994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.680064 17994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.680289 17994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.680385 17994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.680593 17994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:17.680867 17994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:17.680899 17994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.680920 17994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:17.680932 17994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.686514 17994 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:38553
I20260923 15:07:17.686599 18123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:38553 every 8 connection(s)
I20260923 15:07:17.686985 17994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:07:17.687429 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 17994
I20260923 15:07:17.687536 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:07:17.687785 8169 external_mini_cluster.cc:1428] Setting key cd1ca1c306483138933cfa8ab4ce528a
I20260923 15:07:17.688980 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36399
--builtin_ntp_servers=127.7.250.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 15:07:17.690855 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.688163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:60519 (local address 127.7.250.126:36399)
0923 15:07:17.688213 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.688215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.688352 (+ 137us) server_negotiation.cc:407] Connection header received
0923 15:07:17.689036 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.689037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.689045 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.689099 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.690554 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.690556 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.690556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.690561 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.690578 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.690583 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.690670 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.690764 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.690783 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:07:17.691079 18126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.687963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36399 (local address 127.7.250.65:60519)
0923 15:07:17.688218 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:17.688267 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:17.688909 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:17.689229 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:17.689233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:17.689242 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:17.690395 (+ 1153us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:07:17.690396 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:17.690426 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:07:17.690456 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:17.690669 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:17.690676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:17.690680 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:17.690776 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:07:17.690848 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":103,"threads_started":1}
I20260923 15:07:17.691639 18124 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36399
I20260923 15:07:17.691753 18124 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:17.691910 18124 heartbeater.cc:511] Master 127.7.250.126:36399 requested a full tablet report, sending...
I20260923 15:07:17.692484 17933 ts_manager.cc:194] Registered new tserver with Master: 4c102c6e4a844a1bb630ffd8deb415be (127.7.250.65:38553)
W20260923 15:07:17.772949 18127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.773211 18127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.773252 18127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.774471 18127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.774546 18127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.774591 18127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:07:17.775966 18127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.776257 18127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.776567 18127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.782239 18132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.782243 18135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.782243 18133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.782505 18127 server_base.cc:1034] running on GCE node
I20260923 15:07:17.782716 18127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.782946 18127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.784101 18127 hybrid_clock.cc:648] HybridClock initialized: now 1790176037784072 us; error 40 us; skew 500 ppm
I20260923 15:07:17.785561 18127 webserver.cc:460] Webserver started at http://127.7.250.66:39303/ using document root <none> and password file <none>
I20260923 15:07:17.785903 18127 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.785960 18127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.786098 18127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.787242 18127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "c8eed55d0e6744fab54480a668ce3502"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "bb0840c3764366de1bdf1c6ec4bca56d"
server_key_iv: "ab03e35f57016b258bfaac5ae9be03b1"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.787606 18127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "c8eed55d0e6744fab54480a668ce3502"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "bb0840c3764366de1bdf1c6ec4bca56d"
server_key_iv: "ab03e35f57016b258bfaac5ae9be03b1"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.789037 18127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:17.789968 18141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.790295 18127 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:17.790422 18127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "c8eed55d0e6744fab54480a668ce3502"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "bb0840c3764366de1bdf1c6ec4bca56d"
server_key_iv: "ab03e35f57016b258bfaac5ae9be03b1"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.790522 18127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.794008 18127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.794353 18127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.794510 18127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.794749 18127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:17.795073 18127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:17.795130 18127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.795171 18127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:17.795197 18127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.801200 18127 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:39705
I20260923 15:07:17.801241 18256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:39705 every 8 connection(s)
I20260923 15:07:17.801585 18127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:07:17.804849 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.802435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:42215 (local address 127.7.250.126:36399)
0923 15:07:17.802498 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.802500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.802732 (+ 232us) server_negotiation.cc:407] Connection header received
0923 15:07:17.803283 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.803285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.803291 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.803348 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.804586 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.804587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.804588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.804592 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.804608 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.804613 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.804686 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.804760 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.804777 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 15:07:17.805078 18259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.802416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36399 (local address 127.7.250.66:42215)
0923 15:07:17.802606 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:17.802641 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:17.803165 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:17.803433 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:17.803437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:17.803447 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:17.804419 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:07:17.804419 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:17.804448 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:07:17.804477 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:17.804683 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:17.804688 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:17.804692 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:17.804762 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 15:07:17.804851 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260923 15:07:17.805140 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 18127
I20260923 15:07:17.805246 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:07:17.805487 8169 external_mini_cluster.cc:1428] Setting key 91226ae95c694cf431f53644ee968f47
I20260923 15:07:17.805658 18257 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36399
I20260923 15:07:17.805755 18257 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:17.805929 18257 heartbeater.cc:511] Master 127.7.250.126:36399 requested a full tablet report, sending...
I20260923 15:07:17.806373 17933 ts_manager.cc:194] Registered new tserver with Master: c8eed55d0e6744fab54480a668ce3502 (127.7.250.66:39705)
I20260923 15:07:17.806455 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36399
--builtin_ntp_servers=127.7.250.84:44645
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 15:07:17.888674 18260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:17.888944 18260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:17.888983 18260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:17.890192 18260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:17.890269 18260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:17.890355 18260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:07:17.891784 18260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:44645
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:17.892081 18260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:17.892397 18260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:17.898329 18265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.898306 18266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:17.898293 18268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:17.898542 18260 server_base.cc:1034] running on GCE node
I20260923 15:07:17.898834 18260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:17.899075 18260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:17.900242 18260 hybrid_clock.cc:648] HybridClock initialized: now 1790176037900223 us; error 41 us; skew 500 ppm
I20260923 15:07:17.901721 18260 webserver.cc:460] Webserver started at http://127.7.250.67:40797/ using document root <none> and password file <none>
I20260923 15:07:17.902046 18260 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:17.902101 18260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:17.902293 18260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:17.903431 18260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "d038db248ce2462989728af644ae1714"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "3c3f77a8f96f5144512ed47b4f083907"
server_key_iv: "9769ba36fc13cb1fe30762aac3d0c7aa"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.903800 18260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "d038db248ce2462989728af644ae1714"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "3c3f77a8f96f5144512ed47b4f083907"
server_key_iv: "9769ba36fc13cb1fe30762aac3d0c7aa"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.905233 18260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:07:17.906071 18274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.906333 18260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:17.906438 18260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "d038db248ce2462989728af644ae1714"
format_stamp: "Formatted at 2026-09-23 15:07:17 on dist-test-slave-bb3x"
server_key: "3c3f77a8f96f5144512ed47b4f083907"
server_key_iv: "9769ba36fc13cb1fe30762aac3d0c7aa"
server_key_version: "encryptionkey@0"
I20260923 15:07:17.906528 18260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:17.910000 18260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:17.910322 18260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:17.910488 18260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:17.910704 18260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:17.911039 18260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:17.911103 18260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.911141 18260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:17.911170 18260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:17.916921 18260 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:41693
I20260923 15:07:17.916953 18388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:41693 every 8 connection(s)
I20260923 15:07:17.917321 18260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:07:17.920588 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.918090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:51665 (local address 127.7.250.126:36399)
0923 15:07:17.918137 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.918139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.918442 (+ 303us) server_negotiation.cc:407] Connection header received
0923 15:07:17.918991 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.918992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.918999 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.919062 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.920299 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:17.920300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:17.920301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:07:17.920305 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:17.920320 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:07:17.920325 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:17.920405 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:17.920498 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 15:07:17.920516 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:07:17.920792 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:17.918074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36399 (local address 127.7.250.67:51665)
0923 15:07:17.918302 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:17.918345 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:17.918884 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:17.919135 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:17.919139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:17.919149 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:17.920136 (+ 987us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:07:17.920137 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:17.920164 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:07:17.920192 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:17.920403 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:17.920409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:17.920412 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:07:17.920508 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:07:17.920570 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260923 15:07:17.921290 18389 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36399
I20260923 15:07:17.921319 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 18260
I20260923 15:07:17.921391 18389 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:17.921422 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:07:17.921566 18389 heartbeater.cc:511] Master 127.7.250.126:36399 requested a full tablet report, sending...
I20260923 15:07:17.921620 8169 external_mini_cluster.cc:1428] Setting key 16155d82d3457b6e7b04fe516522132d
I20260923 15:07:17.921936 17933 ts_manager.cc:194] Registered new tserver with Master: d038db248ce2462989728af644ae1714 (127.7.250.67:41693)
I20260923 15:07:17.922487 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 15:07:17.924250 17975 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:07:17.923915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60930 (local address 127.7.250.126:36399)
0923 15:07:17.923952 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:17.923954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:17.924015 (+ 61us) server_negotiation.cc:407] Connection header received
0923 15:07:17.924068 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:17.924069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:17.924072 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:17.924118 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:17.924206 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60930: BlockingRecv error: recv got EOF from 127.0.0.1:60930 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:07:17.924335 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17994
I20260923 15:07:17.929847 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 18127
I20260923 15:07:17.934487 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 18260
I20260923 15:07:17.938576 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 17902
2026-09-23T15:07:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (534 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:07:17 dist-test-slave-bb3x krb5kdc[18399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:07:17 dist-test-slave-bb3x krb5kdc[18399](info): set up 2 sockets
Sep 23 15:07:17 dist-test-slave-bb3x krb5kdc[18399](info): commencing operation
krb5kdc: starting...
W20260923 15:07:18.988826 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:07:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:07:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:07:19.119593 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45241
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:45241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:19.203332 18412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:19.203555 18412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:19.203579 18412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:19.204790 18412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:07:19.204833 18412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:19.204849 18412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:19.204861 18412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:07:19.204873 18412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:07:19.206444 18412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:45241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:45241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:19.206698 18412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:19.206928 18412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:19.212141 18420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.212210 18418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:19.212216 18412 server_base.cc:1034] running on GCE node
W20260923 15:07:19.212385 18417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:19.212641 18412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:19.212986 18412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:19.214186 18412 hybrid_clock.cc:648] HybridClock initialized: now 1790176039214176 us; error 33 us; skew 500 ppm
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:19.216765 18412 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:07:19.217242 18412 webserver.cc:460] Webserver started at http://127.7.250.126:37531/ using document root <none> and password file <none>
I20260923 15:07:19.217476 18412 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:19.217520 18412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:19.217671 18412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:19.218809 18412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "580db4b722a440309ad813157139c61c"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "0f8e943644a57f7e730be754cbec29e0"
server_key_iv: "51fadc44d7f1d936b4fafa096393fb0f"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.219177 18412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "580db4b722a440309ad813157139c61c"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "0f8e943644a57f7e730be754cbec29e0"
server_key_iv: "51fadc44d7f1d936b4fafa096393fb0f"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.220534 18412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:07:19.221347 18427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.221547 18412 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:19.221654 18412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "580db4b722a440309ad813157139c61c"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "0f8e943644a57f7e730be754cbec29e0"
server_key_iv: "51fadc44d7f1d936b4fafa096393fb0f"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.221729 18412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:19.232971 18412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:19.235347 18412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:19.235522 18412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:19.239719 18412 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:45241
I20260923 15:07:19.239754 18479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:45241 every 8 connection(s)
I20260923 15:07:19.240113 18412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:07:19.241011 18480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:19.243305 18480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: Bootstrap starting.
I20260923 15:07:19.243857 18480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:19.244095 18480 log.cc:824] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:19.244851 18480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: No bootstrap required, opened a new log
I20260923 15:07:19.244843 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 18412
I20260923 15:07:19.244961 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:07:19.245178 8169 external_mini_cluster.cc:1428] Setting key 25a4be1c6e8f55545921cd7ee1c603ca
I20260923 15:07:19.247375 18480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580db4b722a440309ad813157139c61c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45241 } }
I20260923 15:07:19.247562 18480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:19.247601 18480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580db4b722a440309ad813157139c61c, State: Initialized, Role: FOLLOWER
I20260923 15:07:19.247748 18480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580db4b722a440309ad813157139c61c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45241 } }
I20260923 15:07:19.247838 18480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:07:19.247884 18480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:07:19.247934 18480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:19.248651 18480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580db4b722a440309ad813157139c61c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45241 } }
I20260923 15:07:19.248754 18480 leader_election.cc:302] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 580db4b722a440309ad813157139c61c; no voters:
I20260923 15:07:19.248910 18480 leader_election.cc:288] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:07:19.248972 18485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:07:19.249126 18485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [term 1 LEADER]: Becoming Leader. State: Replica: 580db4b722a440309ad813157139c61c, State: Running, Role: LEADER
I20260923 15:07:19.249135 18480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:07:19.249939 18485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580db4b722a440309ad813157139c61c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45241 } }
I20260923 15:07:19.250583 18486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 580db4b722a440309ad813157139c61c. Latest consensus state: current_term: 1 leader_uuid: "580db4b722a440309ad813157139c61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580db4b722a440309ad813157139c61c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45241 } } }
I20260923 15:07:19.250669 18486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [sys.catalog]: This master's current role is: LEADER
I20260923 15:07:19.250771 18487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "580db4b722a440309ad813157139c61c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580db4b722a440309ad813157139c61c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 45241 } } }
I20260923 15:07:19.251220 18487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c [sys.catalog]: This master's current role is: LEADER
W20260923 15:07:19.251579 18499 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:07:19.251634 18499 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:07:19.251706 18500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:07:19.251896 18500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:19.253955 18500 catalog_manager.cc:1347] Generated new cluster ID: d9a77731dd134764a4551807c1e90a87
I20260923 15:07:19.254009 18500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:07:19.258224 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.245761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34948 (local address 127.7.250.126:45241)
0923 15:07:19.245974 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.245983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.246016 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:07:19.246588 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.246597 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.246661 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.246813 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:19.247509 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.249212 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.250440 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.251376 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.251484 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.254348 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:19.254352 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:19.254352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:19.254365 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:19.256497 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.256872 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.256875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.256876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.256985 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.257197 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.257198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.257199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.257518 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:19.257681 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.257762 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.257936 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":91,"threads_started":1}
I20260923 15:07:19.272429 18500 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:07:19.273221 18500 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:07:19.287616 18500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: Generated new TSK 0
I20260923 15:07:19.287930 18500 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:07:19.325384 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45241
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:19.413359 18507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:19.413564 18507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:19.413590 18507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:19.414803 18507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:19.414847 18507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:19.414873 18507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:07:19.416292 18507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:19.416486 18507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:19.416671 18507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:19.421748 18513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.421833 18515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.421876 18512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:19.422237 18507 server_base.cc:1034] running on GCE node
I20260923 15:07:19.422497 18507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:19.422704 18507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:19.423858 18507 hybrid_clock.cc:648] HybridClock initialized: now 1790176039423838 us; error 32 us; skew 500 ppm
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:19.432083 18507 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:07:19.432559 18507 webserver.cc:460] Webserver started at http://127.7.250.65:38241/ using document root <none> and password file <none>
I20260923 15:07:19.432798 18507 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:19.432879 18507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:19.433037 18507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:19.434110 18507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "57dc50fabc834326b91e94bcb22dac0e"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "993c27158753f0a138103dd843d970c1"
server_key_iv: "acd5cae72d9fd52a1b443efe1b815463"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.434507 18507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "57dc50fabc834326b91e94bcb22dac0e"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "993c27158753f0a138103dd843d970c1"
server_key_iv: "acd5cae72d9fd52a1b443efe1b815463"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.435985 18507 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:19.436823 18523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.437088 18507 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:19.437202 18507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "57dc50fabc834326b91e94bcb22dac0e"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "993c27158753f0a138103dd843d970c1"
server_key_iv: "acd5cae72d9fd52a1b443efe1b815463"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.437283 18507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:19.455802 18507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:19.458104 18507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:19.458370 18507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:19.458657 18507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:19.459003 18507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:19.459079 18507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.459123 18507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:19.459149 18507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.465152 18507 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:36413
I20260923 15:07:19.465193 18638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:36413 every 8 connection(s)
I20260923 15:07:19.465547 18507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:07:19.471356 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 18507
I20260923 15:07:19.471457 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:07:19.471630 8169 external_mini_cluster.cc:1428] Setting key b3160d3fad79da8b123a17f269f35aeb
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:19.475594 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.466322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:54021 (local address 127.7.250.126:45241)
0923 15:07:19.466387 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.466389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.466587 (+ 198us) server_negotiation.cc:407] Connection header received
0923 15:07:19.467099 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.467100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.467113 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.467193 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:19.468157 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.468663 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.469651 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.469933 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.469994 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.472576 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:19.472579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:19.472580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:19.472587 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:19.474083 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.474564 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.474566 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.474566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.474628 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.474907 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.474908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.474909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.475027 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:19.475099 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.475409 (+ 310us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.475502 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:07:19.475808 18641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.466241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45241 (local address 127.7.250.65:54021)
0923 15:07:19.466446 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.466491 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.466969 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.467253 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.467256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.467836 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:19.468046 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.468051 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.468750 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.468751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.469561 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.469564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.470012 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.470013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.470110 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.470661 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:19.470669 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:19.472480 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:19.474186 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:19.474194 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:19.474196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:19.474472 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:19.474705 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:19.474706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:19.474706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:19.474833 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:19.475101 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:19.475109 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:19.475271 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.475376 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.475541 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":94,"threads_started":1}
I20260923 15:07:19.476388 18639 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45241
I20260923 15:07:19.476502 18639 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:19.476687 18639 heartbeater.cc:511] Master 127.7.250.126:45241 requested a full tablet report, sending...
I20260923 15:07:19.477345 18444 ts_manager.cc:194] Registered new tserver with Master: 57dc50fabc834326b91e94bcb22dac0e (127.7.250.65:36413)
I20260923 15:07:19.478341 18444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:54021
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:07:19.497974 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45241
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:19.599771 18644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:19.599980 18644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:19.600004 18644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:19.601110 18644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:19.601150 18644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:19.601176 18644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:07:19.602571 18644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:19.602766 18644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:19.602939 18644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:19.607952 18650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.607941 18649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.607923 18652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:19.608114 18644 server_base.cc:1034] running on GCE node
I20260923 15:07:19.608405 18644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:19.608657 18644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:19.609807 18644 hybrid_clock.cc:648] HybridClock initialized: now 1790176039609788 us; error 38 us; skew 500 ppm
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:19.612397 18644 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:07:19.612761 18644 webserver.cc:460] Webserver started at http://127.7.250.66:33301/ using document root <none> and password file <none>
I20260923 15:07:19.613015 18644 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:19.613060 18644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:19.613191 18644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:19.614306 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "d1e568098edb473187d667c4050b92f6"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "5d0fa8d935e309176661420df07dd835"
server_key_iv: "262c02fbf8b06b671ef3daac7b83e2f8"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.614677 18644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "d1e568098edb473187d667c4050b92f6"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "5d0fa8d935e309176661420df07dd835"
server_key_iv: "262c02fbf8b06b671ef3daac7b83e2f8"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.616056 18644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:19.616880 18659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.617105 18644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:07:19.617190 18644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "d1e568098edb473187d667c4050b92f6"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "5d0fa8d935e309176661420df07dd835"
server_key_iv: "262c02fbf8b06b671ef3daac7b83e2f8"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.617269 18644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:19.640093 18644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:19.642688 18644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:19.642841 18644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:19.643110 18644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:19.643446 18644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:19.643501 18644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.643545 18644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:19.643573 18644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.649538 18644 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:43547
I20260923 15:07:19.649617 18773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:43547 every 8 connection(s)
I20260923 15:07:19.649971 18644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:07:19.654160 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 18644
I20260923 15:07:19.654244 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:07:19.654490 8169 external_mini_cluster.cc:1428] Setting key 772582f31fc9233d4c4b6827da57f21f
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:19.660231 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.650748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:50509 (local address 127.7.250.126:45241)
0923 15:07:19.650828 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.650830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.651103 (+ 273us) server_negotiation.cc:407] Connection header received
0923 15:07:19.651665 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.651666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.651677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.651732 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:19.652729 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.653264 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.654428 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.654685 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.654772 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.657229 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:19.657231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:19.657231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:19.657240 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:19.658790 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.659229 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.659231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.659231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.659299 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.659566 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.659567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.659567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.659688 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:19.659761 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.660056 (+ 295us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.660141 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 15:07:19.660429 18776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.650748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45241 (local address 127.7.250.66:50509)
0923 15:07:19.650993 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.651030 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.651567 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.651860 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.651863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.652436 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:19.652631 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.652636 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.653361 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.653363 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.654336 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.654339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.654789 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.654790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.654957 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.655519 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:19.655527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:19.657138 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:19.658907 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:19.658914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:19.658916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:19.659148 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:19.659376 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:19.659377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:19.659378 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:19.659492 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:19.659770 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:19.659777 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:19.659916 (+ 139us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.660020 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.660185 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":141,"threads_started":1}
I20260923 15:07:19.660983 18774 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45241
I20260923 15:07:19.661096 18774 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:19.661299 18774 heartbeater.cc:511] Master 127.7.250.126:45241 requested a full tablet report, sending...
I20260923 15:07:19.661712 18444 ts_manager.cc:194] Registered new tserver with Master: d1e568098edb473187d667c4050b92f6 (127.7.250.66:43547)
I20260923 15:07:19.662290 18444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:50509
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:07:19.683254 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45241
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:07:19.777878 18779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:07:19.778126 18779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:07:19.778155 18779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:07:19.779520 18779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:07:19.779579 18779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:07:19.779613 18779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:07:19.781201 18779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:45241
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:07:19.781431 18779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:07:19.781636 18779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:07:19.787297 18787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.787272 18784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:07:19.787277 18785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:07:19.787607 18779 server_base.cc:1034] running on GCE node
I20260923 15:07:19.787848 18779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:07:19.788105 18779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:07:19.789309 18779 hybrid_clock.cc:648] HybridClock initialized: now 1790176039789301 us; error 50 us; skew 500 ppm
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:07:19.791855 18779 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:07:19.792317 18779 webserver.cc:460] Webserver started at http://127.7.250.67:35367/ using document root <none> and password file <none>
I20260923 15:07:19.792560 18779 fs_manager.cc:359] Metadata directory not provided
I20260923 15:07:19.792606 18779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:07:19.792752 18779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:07:19.793802 18779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "47b7fbf8ea3a4d6c90e0636102363494"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "6a9dd7ebeb429c38908c564e064b9dc9"
server_key_iv: "06ae725a45e7e623df28ce30361d184d"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.794159 18779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "47b7fbf8ea3a4d6c90e0636102363494"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "6a9dd7ebeb429c38908c564e064b9dc9"
server_key_iv: "06ae725a45e7e623df28ce30361d184d"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.795608 18779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:19.796473 18794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.796650 18779 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:07:19.796736 18779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "47b7fbf8ea3a4d6c90e0636102363494"
format_stamp: "Formatted at 2026-09-23 15:07:19 on dist-test-slave-bb3x"
server_key: "6a9dd7ebeb429c38908c564e064b9dc9"
server_key_iv: "06ae725a45e7e623df28ce30361d184d"
server_key_version: "encryptionkey@0"
I20260923 15:07:19.796792 18779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:07:19.811314 18779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:07:19.813254 18779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:07:19.813370 18779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:07:19.813555 18779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:07:19.813868 18779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:07:19.813901 18779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.813923 18779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:07:19.813969 18779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:07:19.819888 18779 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:35893
I20260923 15:07:19.819948 18910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:35893 every 8 connection(s)
I20260923 15:07:19.820246 18779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:07:19 dist-test-slave-bb3x krb5kdc[18399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176039, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:07:19.829227 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 18779
I20260923 15:07:19.829375 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:07:19.829574 8169 external_mini_cluster.cc:1428] Setting key 40b7fdc1c168b612baa67c642c61b7e3
I20260923 15:07:19.829579 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.821043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:38857 (local address 127.7.250.126:45241)
0923 15:07:19.821085 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.821087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.821292 (+ 205us) server_negotiation.cc:407] Connection header received
0923 15:07:19.821906 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.821907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.821919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.821985 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:19.822978 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.823483 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.824561 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.824751 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.824837 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.827015 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:19.827016 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:19.827016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:19.827024 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:19.828203 (+ 1179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.828566 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.828566 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.828567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.828618 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.828859 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.828861 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.828861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.828984 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:19.829052 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.829398 (+ 346us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.829485 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:07:19.829779 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.821003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:45241 (local address 127.7.250.67:38857)
0923 15:07:19.821165 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.821202 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.821776 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.822054 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.822058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.822679 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:07:19.822873 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.822878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.823578 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.823579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.824461 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.824463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.824933 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.824934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.825042 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.825522 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:07:19.825530 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:07:19.826928 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:07:19.828275 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:19.828281 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:19.828283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:19.828500 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:19.828677 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:07:19.828677 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:07:19.828678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:07:19.828771 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:07:19.829091 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:07:19.829098 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:07:19.829216 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.829372 (+ 156us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.829535 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260923 15:07:19.830322 18911 heartbeater.cc:347] Connected to a master server at 127.7.250.126:45241
I20260923 15:07:19.830430 18911 heartbeater.cc:464] Registering TS with master...
I20260923 15:07:19.830626 18911 heartbeater.cc:511] Master 127.7.250.126:45241 requested a full tablet report, sending...
I20260923 15:07:19.831004 18443 ts_manager.cc:194] Registered new tserver with Master: 47b7fbf8ea3a4d6c90e0636102363494 (127.7.250.67:35893)
I20260923 15:07:19.831398 18443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:38857
I20260923 15:07:19.841694 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:07:19.849014 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.843130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.7.250.126:45241)
0923 15:07:19.843191 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.843193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.843231 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:07:19.843296 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.843296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.843305 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.843364 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:07:19.843892 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.844367 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.845173 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.845371 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.845438 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.846690 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:07:19.846691 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:07:19.846691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:07:19.846697 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:07:19.847978 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.848302 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.848302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.848303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.848353 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:07:19.848512 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:07:19.848512 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:07:19.848512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:07:19.848652 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:07:19.848726 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.848885 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.848948 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:07:19.850958 18443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:07:19.852502 18443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:07:19.859679 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.854966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.7.250.66:43547)
0923 15:07:19.855406 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.855412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.855422 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:19.855465 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.855466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.855481 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.855593 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:07:19.855767 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.856953 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.858306 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.859276 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.859371 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.859457 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.859565 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.859604 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":107,"threads_started":1}
I20260923 15:07:19.859687 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.854826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:43547 (local address 127.0.0.1:47086)
0923 15:07:19.855100 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.855143 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.855215 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.855605 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.855609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.855611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:07:19.855683 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.855684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.857061 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.857062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.858175 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.858178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.859369 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.859370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.859463 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.859478 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.859552 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.859603 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":80,"threads_started":1}
I20260923 15:07:19.860198 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.855177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47052 (local address 127.7.250.67:35893)
0923 15:07:19.855506 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.855510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.855520 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:07:19.855559 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.855561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.855578 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.855652 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:07:19.855907 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.856973 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.859122 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.859832 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.859899 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.859959 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.860085 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.860125 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":101,"threads_started":1}
I20260923 15:07:19.860296 18922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.854928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:35893 (local address 127.0.0.1:47052)
0923 15:07:19.855168 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.855179 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.855232 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.855723 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.855724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.855726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:07:19.855826 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.855828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.858305 (+ 2477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.858306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.859041 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.859042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.859912 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.859914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.859989 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.859994 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.860072 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.860115 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":106,"threads_started":1}
I20260923 15:07:19.860495 18707 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2314259c33400799aaf2da85b70716 (DEFAULT_TABLE table=test-table [id=f6ee40b62d1e4ff5bd3b8ab6ba48a890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:07:19.860855 18707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2314259c33400799aaf2da85b70716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:19.861119 18843 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2314259c33400799aaf2da85b70716 (DEFAULT_TABLE table=test-table [id=f6ee40b62d1e4ff5bd3b8ab6ba48a890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:07:19.861482 18843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2314259c33400799aaf2da85b70716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:19.862675 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.855044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:36413 (local address 127.0.0.1:37736)
0923 15:07:19.855741 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.855751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.855798 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.855978 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.855978 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.855980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:07:19.856023 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.856024 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.856847 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.856848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.858175 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.858178 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.862419 (+ 4241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.862420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.862485 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.862489 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.862549 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.862603 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":5,"thread_start_us":67,"threads_started":1}
I20260923 15:07:19.862987 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.855253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37736 (local address 127.7.250.65:36413)
0923 15:07:19.855437 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.855441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.855855 (+ 414us) server_negotiation.cc:407] Connection header received
0923 15:07:19.855898 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.855900 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.855914 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.855984 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:07:19.856078 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.856747 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.858310 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.859276 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.859371 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.859452 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.862856 (+ 3404us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.862909 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260923 15:07:19.863679 18928 tablet_bootstrap.cc:492] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Bootstrap starting.
I20260923 15:07:19.864111 18572 tablet_service.cc:1467] Processing CreateTablet for tablet 2d2314259c33400799aaf2da85b70716 (DEFAULT_TABLE table=test-table [id=f6ee40b62d1e4ff5bd3b8ab6ba48a890]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:07:19.864362 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d2314259c33400799aaf2da85b70716. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:07:19.864603 18928 tablet_bootstrap.cc:654] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:19.865327 18929 tablet_bootstrap.cc:492] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Bootstrap starting.
I20260923 15:07:19.865689 18928 log.cc:824] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:19.866029 18929 tablet_bootstrap.cc:654] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:19.866573 18928 tablet_bootstrap.cc:492] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: No bootstrap required, opened a new log
I20260923 15:07:19.866612 18929 log.cc:824] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:19.866669 18928 ts_tablet_manager.cc:1397] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:07:19.867419 18929 tablet_bootstrap.cc:492] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: No bootstrap required, opened a new log
I20260923 15:07:19.867497 18929 ts_tablet_manager.cc:1397] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:07:19.867455 18932 tablet_bootstrap.cc:492] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: Bootstrap starting.
I20260923 15:07:19.868306 18932 tablet_bootstrap.cc:654] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: Neither blocks nor log segments found. Creating new log.
I20260923 15:07:19.868952 18932 log.cc:824] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:07:19.869164 18929 raft_consensus.cc:348] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.869340 18929 raft_consensus.cc:374] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:19.869385 18929 raft_consensus.cc:729] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b7fbf8ea3a4d6c90e0636102363494, State: Initialized, Role: FOLLOWER
I20260923 15:07:19.869287 18928 raft_consensus.cc:348] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.869539 18928 raft_consensus.cc:374] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:19.869515 18929 consensus_queue.cc:260] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.869592 18928 raft_consensus.cc:729] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1e568098edb473187d667c4050b92f6, State: Initialized, Role: FOLLOWER
I20260923 15:07:19.869748 18928 consensus_queue.cc:260] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.869805 18932 tablet_bootstrap.cc:492] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: No bootstrap required, opened a new log
I20260923 15:07:19.869817 18929 ts_tablet_manager.cc:1428] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:07:19.869843 18911 heartbeater.cc:503] Master 127.7.250.126:45241 was elected leader, sending a full tablet report...
I20260923 15:07:19.869869 18932 ts_tablet_manager.cc:1397] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:07:19.870086 18774 heartbeater.cc:503] Master 127.7.250.126:45241 was elected leader, sending a full tablet report...
I20260923 15:07:19.870071 18928 ts_tablet_manager.cc:1428] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260923 15:07:19.871693 18932 raft_consensus.cc:348] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.871836 18932 raft_consensus.cc:374] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:07:19.871881 18932 raft_consensus.cc:729] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57dc50fabc834326b91e94bcb22dac0e, State: Initialized, Role: FOLLOWER
I20260923 15:07:19.871990 18932 consensus_queue.cc:260] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.872241 18932 ts_tablet_manager.cc:1428] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:07:19.872301 18639 heartbeater.cc:503] Master 127.7.250.126:45241 was elected leader, sending a full tablet report...
W20260923 15:07:19.900617 18775 tablet.cc:2369] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:07:19.966361 18640 tablet.cc:2369] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:07:19.983083 18934 raft_consensus.cc:482] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:07:19.983220 18934 raft_consensus.cc:504] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.983620 18934 leader_election.cc:288] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1e568098edb473187d667c4050b92f6 (127.7.250.66:43547), 57dc50fabc834326b91e94bcb22dac0e (127.7.250.65:36413)
I20260923 15:07:19.986955 18913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.983919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:43547 (local address 127.7.250.67:33723)
0923 15:07:19.983990 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.984005 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.984092 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.984359 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.984360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.984362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:07:19.984447 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.984450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.985190 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.985191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.986010 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.986012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.986679 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.986680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.986774 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.986777 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.986831 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.986877 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:07:19.986956 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.984162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:57517 (local address 127.7.250.65:36413)
0923 15:07:19.984231 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.984234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.984349 (+ 115us) server_negotiation.cc:407] Connection header received
0923 15:07:19.984399 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.984399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.984409 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.984466 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:07:19.984659 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.985099 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.986158 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.986572 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.986631 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.986661 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.986844 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.986888 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:07:19.986964 18937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.984076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:36413 (local address 127.7.250.67:57517)
0923 15:07:19.984267 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:19.984277 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:07:19.984353 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:07:19.984498 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:07:19.984499 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:07:19.984501 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:07:19.984589 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.984591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.985188 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.985189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.986035 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:07:19.986037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.986650 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.986652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:07:19.986739 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.986743 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:07:19.986824 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 15:07:19.986867 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":115,"threads_started":1}
I20260923 15:07:19.987259 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:19.984055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:33723 (local address 127.7.250.66:43547)
0923 15:07:19.984149 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:19.984151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:19.984160 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:07:19.984211 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:19.984212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:19.984222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:19.984283 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:07:19.984535 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.985096 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.986113 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:19.986580 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:19.986670 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:19.986679 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:19.987122 (+ 443us) server_negotiation.cc:299] Negotiation successful
0923 15:07:19.987172 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 15:07:19.987473 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2314259c33400799aaf2da85b70716" candidate_uuid: "47b7fbf8ea3a4d6c90e0636102363494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57dc50fabc834326b91e94bcb22dac0e" is_pre_election: true
I20260923 15:07:19.987531 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2314259c33400799aaf2da85b70716" candidate_uuid: "47b7fbf8ea3a4d6c90e0636102363494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1e568098edb473187d667c4050b92f6" is_pre_election: true
I20260923 15:07:19.987669 18592 raft_consensus.cc:2454] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b7fbf8ea3a4d6c90e0636102363494 in term 0.
I20260923 15:07:19.987713 18727 raft_consensus.cc:2454] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 47b7fbf8ea3a4d6c90e0636102363494 in term 0.
I20260923 15:07:19.987977 18795 leader_election.cc:302] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b7fbf8ea3a4d6c90e0636102363494, 57dc50fabc834326b91e94bcb22dac0e; no voters:
I20260923 15:07:19.988134 18934 raft_consensus.cc:2781] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:07:19.988207 18934 raft_consensus.cc:482] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:07:19.988226 18934 raft_consensus.cc:3037] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:19.988978 18934 raft_consensus.cc:504] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.989089 18934 leader_election.cc:288] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [CANDIDATE]: Term 1 election: Requested vote from peers d1e568098edb473187d667c4050b92f6 (127.7.250.66:43547), 57dc50fabc834326b91e94bcb22dac0e (127.7.250.65:36413)
I20260923 15:07:19.989279 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2314259c33400799aaf2da85b70716" candidate_uuid: "47b7fbf8ea3a4d6c90e0636102363494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1e568098edb473187d667c4050b92f6"
I20260923 15:07:19.989329 18592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d2314259c33400799aaf2da85b70716" candidate_uuid: "47b7fbf8ea3a4d6c90e0636102363494" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57dc50fabc834326b91e94bcb22dac0e"
I20260923 15:07:19.989369 18727 raft_consensus.cc:3037] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:19.989439 18592 raft_consensus.cc:3037] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:07:19.990108 18727 raft_consensus.cc:2454] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b7fbf8ea3a4d6c90e0636102363494 in term 1.
I20260923 15:07:19.990298 18795 leader_election.cc:302] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 47b7fbf8ea3a4d6c90e0636102363494, d1e568098edb473187d667c4050b92f6; no voters:
I20260923 15:07:19.990320 18592 raft_consensus.cc:2454] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 47b7fbf8ea3a4d6c90e0636102363494 in term 1.
I20260923 15:07:19.990475 18934 raft_consensus.cc:2781] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:07:19.990651 18934 raft_consensus.cc:686] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 1 LEADER]: Becoming Leader. State: Replica: 47b7fbf8ea3a4d6c90e0636102363494, State: Running, Role: LEADER
I20260923 15:07:19.990758 18934 consensus_queue.cc:237] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } }
I20260923 15:07:19.991653 18443 catalog_manager.cc:5495] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 reported cstate change: term changed from 0 to 1, leader changed from <none> to 47b7fbf8ea3a4d6c90e0636102363494 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "47b7fbf8ea3a4d6c90e0636102363494" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47b7fbf8ea3a4d6c90e0636102363494" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35893 } health_report { overall_health: HEALTHY } } }
I20260923 15:07:20.008893 18926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:20.006143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47056 (local address 127.7.250.67:35893)
0923 15:07:20.006222 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 15:07:20.006224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:07:20.006242 (+ 18us) server_negotiation.cc:407] Connection header received
0923 15:07:20.006337 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:07:20.006338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:07:20.006367 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:07:20.006437 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:07:20.006638 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:20.007161 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:20.007984 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:07:20.008181 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:07:20.008270 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:07:20.008468 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:07:20.008579 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:07:20.008668 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:07:20.008686 (+ 18us) server_negotiation.cc:299] Negotiation successful
0923 15:07:20.008705 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 15:07:20.010866 18727 raft_consensus.cc:1261] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 1 FOLLOWER]: Refusing update from remote peer 47b7fbf8ea3a4d6c90e0636102363494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:07:20.010910 18592 raft_consensus.cc:1261] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 1 FOLLOWER]: Refusing update from remote peer 47b7fbf8ea3a4d6c90e0636102363494: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:07:20.011310 18934 consensus_queue.cc:1035] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1e568098edb473187d667c4050b92f6" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:07:20.011497 18938 consensus_queue.cc:1035] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57dc50fabc834326b91e94bcb22dac0e" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 36413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:07:20.014856 18942 mvcc.cc:205] Tried to move back new op lower bound from 7332561059881308160 to 7332561059803049984. Current Snapshot: MvccSnapshot[applied={T|T < 7332561059881308160}]
I20260923 15:07:20.017668 18442 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:07:20.017766 18442 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:07:20.018641 18442 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 580db4b722a440309ad813157139c61c: Sending DeleteTablet for 3 replicas of tablet 2d2314259c33400799aaf2da85b70716
I20260923 15:07:20.019011 18843 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d2314259c33400799aaf2da85b70716 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:07:20 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:47052
I20260923 15:07:20.019086 18707 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d2314259c33400799aaf2da85b70716 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:07:20 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:47086
I20260923 15:07:20.019124 18572 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d2314259c33400799aaf2da85b70716 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:07:20 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37736
I20260923 15:07:20.019392 18951 tablet_replica.cc:330] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: stopping tablet replica
I20260923 15:07:20.019387 18953 tablet_replica.cc:330] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e: stopping tablet replica
I20260923 15:07:20.019497 18953 raft_consensus.cc:2229] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:07:20.019520 18951 raft_consensus.cc:2229] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:07:20.019645 18951 raft_consensus.cc:2258] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:07:20.019675 18953 raft_consensus.cc:2258] T 2d2314259c33400799aaf2da85b70716 P 57dc50fabc834326b91e94bcb22dac0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:07:20.019675 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 18507
I20260923 15:07:20.020605 18951 ts_tablet_manager.cc:1905] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:07:20.021960 18952 tablet_replica.cc:330] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: stopping tablet replica
I20260923 15:07:20.022099 18952 raft_consensus.cc:2229] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:07:20.022212 18952 raft_consensus.cc:2258] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:07:20.022369 18951 ts_tablet_manager.cc:1918] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:07:20.022477 18951 log.cc:1196] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/2d2314259c33400799aaf2da85b70716
I20260923 15:07:20.022830 18951 ts_tablet_manager.cc:1939] T 2d2314259c33400799aaf2da85b70716 P 47b7fbf8ea3a4d6c90e0636102363494: Deleting consensus metadata
I20260923 15:07:20.022831 18952 ts_tablet_manager.cc:1905] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:07:20.023329 18429 catalog_manager.cc:4841] TS 47b7fbf8ea3a4d6c90e0636102363494 (127.7.250.67:35893): tablet 2d2314259c33400799aaf2da85b70716 (table test-table [id=f6ee40b62d1e4ff5bd3b8ab6ba48a890]) successfully deleted
I20260923 15:07:20.024628 18952 ts_tablet_manager.cc:1918] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:07:20.024712 18952 log.cc:1196] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/2d2314259c33400799aaf2da85b70716
I20260923 15:07:20.024991 18952 ts_tablet_manager.cc:1939] T 2d2314259c33400799aaf2da85b70716 P d1e568098edb473187d667c4050b92f6: Deleting consensus metadata
I20260923 15:07:20.025431 18428 catalog_manager.cc:4841] TS d1e568098edb473187d667c4050b92f6 (127.7.250.66:43547): tablet 2d2314259c33400799aaf2da85b70716 (table test-table [id=f6ee40b62d1e4ff5bd3b8ab6ba48a890]) successfully deleted
W20260923 15:07:20.026374 18428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:36413 (error 108)
I20260923 15:07:20.026544 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 18644
I20260923 15:07:20.027619 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:07:20.027488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:36413 (local address 127.0.0.1:37750)
0923 15:07:20.027544 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 15:07:20.027581 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:36413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:07:20.027729 18428 catalog_manager.cc:4584] TS 57dc50fabc834326b91e94bcb22dac0e (127.7.250.65:36413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2d2314259c33400799aaf2da85b70716: Network error: Client connection negotiation failed: client connection to 127.7.250.65:36413: connect: Connection refused (error 111)
I20260923 15:07:20.033656 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 18779
I20260923 15:07:20.038955 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 18412
2026-09-23T15:07:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2102 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:07:20 dist-test-slave-bb3x krb5kdc[18957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:07:20 dist-test-slave-bb3x krb5kdc[18957](info): set up 2 sockets
Sep 23 15:07:20 dist-test-slave-bb3x krb5kdc[18957](info): commencing operation
krb5kdc: starting...
W20260923 15:07:21.101917 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:07:21 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176041, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:07:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/rangeradmin_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/rangeradmin_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/rangerlookup_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/rangerlookup_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
I20260923 15:07:21.289542 8169 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOOn7ma/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/postgres -l logfile start
2026-09-23 15:07:24.218 UTC [18988] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 15:07:24.218 UTC [18988] LOG: listening on IPv4 address "127.7.250.84", port 50097
2026-09-23 15:07:24.222 UTC [18988] LOG: listening on Unix socket "/tmp/.s.PGSQL.50097"
2026-09-23 15:07:24.244 UTC [18991] LOG: database system was shut down at 2026-09-23 15:07:22 UTC
2026-09-23 15:07:24.248 UTC [18988] LOG: database system is ready to accept connections
I20260923 15:07:25.257398 8169 mini_postgres.cc:96] Postgres bound to 50097
2026-09-23 15:07:25.264 UTC [18999] FATAL: database "slave" does not exist
127.7.250.84:50097 - accepting connections
I20260923 15:07:25.265506 8169 mini_ranger.cc:161] Starting Ranger...
I20260923 15:07:25.313323 8169 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 15:07:25.917351 8169 mini_ranger.cc:87] Created ranger Postgres database
I20260923 15:07:25.917464 8169 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-admin
2026-09-23 15:07:26,453 [I] DB FLAVOR :POSTGRES
2026-09-23 15:07:26,454 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:07:26,454 [I] Checking connection..
2026-09-23 15:07:26,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 15:07:26,745 [I] Checking connection passed.
2026-09-23 15:07:26,745 [I] --------- Verifying version history table ---------
2026-09-23 15:07:26,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:07:26,996 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:07:26,996 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:07:27,248 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:07:27,249 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 15:07:27,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 15:07:27.484 UTC [19063] WARNING: there is no transaction in progress
2026-09-23 15:07:27,491 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 15:07:27,491 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:07:27,740 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 15:07:27,741 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:07:27,741 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 15:07:27,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 15:07:28,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bb3x.c.gcp-upstream.internal','N') ;"
2026-09-23 15:07:28,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:07:28,763 [I] Table x_portal_user does not exist in database ranger
2026-09-23 15:07:28,763 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:07:29,019 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 15:07:29,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:07:29,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:07:29,510 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 15:07:29,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 15:07:30.618 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.638 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.657 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.674 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.696 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.716 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.779 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.787 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.805 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.818 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.830 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.845 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.856 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.864 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.873 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:30.881 UTC [19174] WARNING: there is no transaction in progress
2026-09-23 15:07:31,681 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 15:07:31,682 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 15:07:31,930 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 15:07:31,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:07:32,178 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 15:07:32,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:07:32,426 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 15:07:32,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:07:32,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:07:32,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bb3x.c.gcp-upstream.internal';"
2026-09-23 15:07:33,161 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 15:07:33,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 15:07:33,414 [I] DB_PATCHES have already been applied
I20260923 15:07:33.420665 8169 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOOn7ma/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 15:07:33.420737 8169 mini_ranger.cc:193] Using host: 127.7.250.84
I20260923 15:07:33.424219 8169 mini_ranger.cc:227] Ranger admin URL: http://127.7.250.84:36523
Sep 23, 2026 3:07:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:07:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:07:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 3:07:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 3:07:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 3:07:34 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 3:07:34 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 15:07:35.679646 8169 mini_ranger.cc:160] Time spent starting Ranger: real 10.414s user 0.000s sys 0.011s
Sep 23, 2026 3:07:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 3:07:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 3:07:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 3:07:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 3:07:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 15:07:59.617671 8169 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/rangerkms_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/rangerkms_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.250.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.250.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 15:07:59 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176079, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 15:07:59.704579 8169 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 15:07:59.714828 8169 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 15:08:00.729771 8169 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 15:08:00.729872 8169 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms
I20260923 15:08:00.729899 8169 mini_ranger_kms.cc:227] Using postgres at 127.7.250.84:50097
2026-09-23 15:08:01,363 [I] DB FLAVOR :POSTGRES
2026-09-23 15:08:01,363 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:08:01,363 [I] Checking connection
2026-09-23 15:08:01,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 15:08:01,623 [I] connection success
2026-09-23 15:08:01,623 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 15:08:01,623 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 15:08:01,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 15:08:01,893 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 15:08:01,893 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:08:01,894 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 15:08:01,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:50097/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 15:08:02,188 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 15:08:02.686662 8169 mini_ranger_kms.cc:310] Created kms service
I20260923 15:08:02.897086 8169 mini_ranger_kms.cc:326] Created kudu user
I20260923 15:08:02.944160 8169 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 15:08:02 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176079, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:03.386452 8169 mini_ranger_kms.cc:384] Added ranger policy
I20260923 15:08:03.386575 8169 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 15:08:03.386600 8169 mini_ranger_kms.cc:240] Using host: 127.7.250.84
I20260923 15:08:03.390136 8169 mini_ranger_kms.cc:276] Ranger KMS PID: 19359
I20260923 15:08:03.390302 8169 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.250.84:49681
Sep 23, 2026 3:08:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:08:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:08:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 3:08:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 3:08:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 3:08:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 15:08:05.746127 8169 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.042s user 0.007s sys 0.248s
I20260923 15:08:05.746248 8169 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 15:08:05.746359 8169 mini_ranger_kms.cc:401] 127.7.250.84:49681/kms/v1/keys
Sep 23 15:08:06 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790176086, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.250.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 15:08:06 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176086, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.250.84@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
Sep 23 15:08:06 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176086, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.250.84@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
Sep 23 15:08:09 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:08:09.369728 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42121
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:42121
--ranger_config_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:49681/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:09.472537 19410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:09.472733 19410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:09.472756 19410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:09.473994 19410 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:08:09.474036 19410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:09.474050 19410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:09.474062 19410 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:08:09.474074 19410 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:08:09.475737 19410 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.250.84:49681/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:42121
--ranger_config_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:09.475976 19410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:09.476202 19410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:09.483485 19416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:09.483546 19410 server_base.cc:1034] running on GCE node
W20260923 15:08:09.483564 19415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:09.483703 19418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:09.483982 19410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:09.484342 19410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:09.485567 19410 hybrid_clock.cc:648] HybridClock initialized: now 1790176089485531 us; error 32 us; skew 500 ppm
Sep 23 15:08:09 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:09.493438 19410 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:08:09.493965 19410 webserver.cc:460] Webserver started at http://127.7.250.126:43733/ using document root <none> and password file <none>
I20260923 15:08:09.494207 19410 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:09.494289 19410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:09.494465 19410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:09 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176089, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:09.563905 19410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "64004eb4b94545d2b7da5db46e47e85b"
format_stamp: "Formatted at 2026-09-23 15:08:09 on dist-test-slave-bb3x"
server_key: "c6893f771bdcaccf3ec80686ec71c29f"
server_key_iv: "81986dfe3e46522f79b090734b01858a"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:09.564455 19410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "64004eb4b94545d2b7da5db46e47e85b"
format_stamp: "Formatted at 2026-09-23 15:08:09 on dist-test-slave-bb3x"
server_key: "c6893f771bdcaccf3ec80686ec71c29f"
server_key_iv: "81986dfe3e46522f79b090734b01858a"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:09.565955 19410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:08:09.581287 19425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:09.581771 19410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:08:09.581897 19410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "64004eb4b94545d2b7da5db46e47e85b"
format_stamp: "Formatted at 2026-09-23 15:08:09 on dist-test-slave-bb3x"
server_key: "c6893f771bdcaccf3ec80686ec71c29f"
server_key_iv: "81986dfe3e46522f79b090734b01858a"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:09.582036 19410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:09.597959 19410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:09.599817 19410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:09.600018 19410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:09.606734 19410 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:42121
I20260923 15:08:09.606791 19477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:42121 every 8 connection(s)
I20260923 15:08:09.607165 19410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:08:09.608270 19478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:08:09.611775 19478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b: Bootstrap starting.
I20260923 15:08:09.612627 19478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b: Neither blocks nor log segments found. Creating new log.
I20260923 15:08:09.613603 19478 log.cc:824] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:08:09.614526 19478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b: No bootstrap required, opened a new log
I20260923 15:08:09.616716 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 19410
I20260923 15:08:09.616860 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:08:09.617055 8169 external_mini_cluster.cc:1428] Setting key eca3155d31f686e514e22cacc65be8b5
I20260923 15:08:09.618940 19478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64004eb4b94545d2b7da5db46e47e85b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42121 } }
I20260923 15:08:09.619207 19478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:08:09.619330 19478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64004eb4b94545d2b7da5db46e47e85b, State: Initialized, Role: FOLLOWER
I20260923 15:08:09.619513 19478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64004eb4b94545d2b7da5db46e47e85b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42121 } }
I20260923 15:08:09.619601 19478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:08:09.619630 19478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:08:09.619699 19478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:08:09.620440 19478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64004eb4b94545d2b7da5db46e47e85b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42121 } }
I20260923 15:08:09.620546 19478 leader_election.cc:302] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64004eb4b94545d2b7da5db46e47e85b; no voters:
I20260923 15:08:09.620745 19478 leader_election.cc:288] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:08:09.621061 19478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:08:09.621374 19483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:08:09.621613 19483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [term 1 LEADER]: Becoming Leader. State: Replica: 64004eb4b94545d2b7da5db46e47e85b, State: Running, Role: LEADER
I20260923 15:08:09.621872 19483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64004eb4b94545d2b7da5db46e47e85b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42121 } }
I20260923 15:08:09.623052 19484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64004eb4b94545d2b7da5db46e47e85b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64004eb4b94545d2b7da5db46e47e85b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42121 } } }
I20260923 15:08:09.623134 19484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [sys.catalog]: This master's current role is: LEADER
I20260923 15:08:09.623611 19478 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 15:08:09.624189 19498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:08:09.624225 19485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64004eb4b94545d2b7da5db46e47e85b. Latest consensus state: current_term: 1 leader_uuid: "64004eb4b94545d2b7da5db46e47e85b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64004eb4b94545d2b7da5db46e47e85b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42121 } } }
I20260923 15:08:09.624400 19485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b [sys.catalog]: This master's current role is: LEADER
Sep 23 15:08:09 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:09.630439 19498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:08:09.633445 19482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:09.617582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53464 (local address 127.7.250.126:42121)
0923 15:08:09.617792 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:09.617803 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:09.617836 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:08:09.618920 (+ 1084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:09.618964 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:09.619022 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:09.619188 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:09.620007 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:09.622202 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:09.623417 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:09.624300 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:09.624410 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:09.627621 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:09.627656 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:09.627657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:09.627730 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:09.629857 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:09.630228 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:09.630231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:09.630232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:09.630390 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:09.630778 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:09.630779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:09.630780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:09.631408 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:09.631616 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:09.632384 (+ 768us) server_negotiation.cc:299] Negotiation successful
0923 15:08:09.632835 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260923 15:08:09.637595 19498 catalog_manager.cc:1347] Generated new cluster ID: 72773d8ce7e44eacacad2cf39dcd28f2
I20260923 15:08:09.637650 19498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:08:09.677022 19498 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:08:09.678207 19498 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:08:09.689462 19498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b: Generated new TSK 0
I20260923 15:08:09.689808 19498 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:08:10.570083 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:49681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42121
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:10.670657 19524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:10.670886 19524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:10.670924 19524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:10.672093 19524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:10.672163 19524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:10.672206 19524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:08:10.673797 19524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.250.84:49681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:10.674075 19524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:10.674360 19524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:10.680594 19530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:10.680662 19529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:10.680853 19532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:10.681394 19524 server_base.cc:1034] running on GCE node
I20260923 15:08:10.681649 19524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:10.681921 19524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:10.683107 19524 hybrid_clock.cc:648] HybridClock initialized: now 1790176090683083 us; error 40 us; skew 500 ppm
Sep 23 15:08:10 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:10.685832 19524 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:08:10.686465 19524 webserver.cc:460] Webserver started at http://127.7.250.65:39483/ using document root <none> and password file <none>
I20260923 15:08:10.686722 19524 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:10.686767 19524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:10.686928 19524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:10 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:10.702847 19524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "4b07303d36b4430299f6e86bf5c35dcd"
format_stamp: "Formatted at 2026-09-23 15:08:10 on dist-test-slave-bb3x"
server_key: "10f054214664670508f7c656b9688dc2"
server_key_iv: "cfe9dae514992070513a0c005fb4182a"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:10.703691 19524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "4b07303d36b4430299f6e86bf5c35dcd"
format_stamp: "Formatted at 2026-09-23 15:08:10 on dist-test-slave-bb3x"
server_key: "10f054214664670508f7c656b9688dc2"
server_key_iv: "cfe9dae514992070513a0c005fb4182a"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:10.707664 19524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260923 15:08:10.723822 19541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:10.724076 19524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260923 15:08:10.724179 19524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "4b07303d36b4430299f6e86bf5c35dcd"
format_stamp: "Formatted at 2026-09-23 15:08:10 on dist-test-slave-bb3x"
server_key: "10f054214664670508f7c656b9688dc2"
server_key_iv: "cfe9dae514992070513a0c005fb4182a"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:10.724278 19524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:10.739391 19524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:10.741927 19524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:10.742113 19524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:10.742445 19524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:10.742827 19524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:10.742877 19524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:10.742909 19524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:10.742926 19524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:10.759224 19524 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:37523
I20260923 15:08:10.759274 19654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:37523 every 8 connection(s)
I20260923 15:08:10.759858 19524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:08:10.767349 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 19524
I20260923 15:08:10.767457 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:08:10.767661 8169 external_mini_cluster.cc:1428] Setting key 3ada7e0b6c4e4d2f22ddec7c9342a7e8
Sep 23 15:08:10 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:10.772485 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:10.760805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:58865 (local address 127.7.250.126:42121)
0923 15:08:10.760955 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:10.760957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:10.761331 (+ 374us) server_negotiation.cc:407] Connection header received
0923 15:08:10.762326 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:10.762327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:10.762342 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:10.762395 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:10.763408 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:10.764242 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:10.765840 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:10.766135 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:10.766182 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:10.768740 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:10.768742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:10.768742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:10.768750 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:10.770738 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:10.771234 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:10.771235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:10.771236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:10.771304 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:10.771624 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:10.771625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:10.771626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:10.771784 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:10.771854 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:10.772216 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 15:08:10.772343 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260923 15:08:10.772625 19658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:10.760797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42121 (local address 127.7.250.65:58865)
0923 15:08:10.761122 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:10.761184 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:10.762169 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:10.762599 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:10.762604 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:10.762925 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:10.763251 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:10.763258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:10.764392 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:10.764393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:10.765728 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:10.765731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:10.766305 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:10.766306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:10.766461 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:10.767106 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:10.767116 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:10.768627 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:10.770835 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:10.770843 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:10.770846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:10.771143 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:10.771400 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:10.771401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:10.771402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:10.771546 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:10.771866 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:10.771873 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:10.772046 (+ 173us) client_negotiation.cc:769] Sending connection context
0923 15:08:10.772176 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 15:08:10.772349 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":193,"threads_started":1}
I20260923 15:08:10.773461 19655 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42121
I20260923 15:08:10.773582 19655 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:10.773804 19655 heartbeater.cc:511] Master 127.7.250.126:42121 requested a full tablet report, sending...
I20260923 15:08:10.774514 19442 ts_manager.cc:194] Registered new tserver with Master: 4b07303d36b4430299f6e86bf5c35dcd (127.7.250.65:37523)
I20260923 15:08:10.775553 19442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:58865
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:08:10.798162 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:49681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42121
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:10.911659 19661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:10.911880 19661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:10.911919 19661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:10.913122 19661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:10.913192 19661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:10.913237 19661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:08:10.915019 19661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.250.84:49681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:10.915292 19661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:10.915572 19661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:10.921554 19671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:10.921753 19669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:10.922017 19661 server_base.cc:1034] running on GCE node
W20260923 15:08:10.922415 19668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:10.922676 19661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:10.922962 19661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:10.926244 19661 hybrid_clock.cc:648] HybridClock initialized: now 1790176090926215 us; error 41 us; skew 500 ppm
Sep 23 15:08:10 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:10.929701 19661 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:08:10.930223 19661 webserver.cc:460] Webserver started at http://127.7.250.66:42337/ using document root <none> and password file <none>
I20260923 15:08:10.930538 19661 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:10.930608 19661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:10.930799 19661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:10 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 15:08:10 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:10.946491 19661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "5207943237114e71b7375daa1dc3fe5e"
format_stamp: "Formatted at 2026-09-23 15:08:10 on dist-test-slave-bb3x"
server_key: "0750dfe173edde643aa29de588d3a0b7"
server_key_iv: "5444d236ad93ff0d4afc55725266ec3c"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:10.947067 19661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "5207943237114e71b7375daa1dc3fe5e"
format_stamp: "Formatted at 2026-09-23 15:08:10 on dist-test-slave-bb3x"
server_key: "0750dfe173edde643aa29de588d3a0b7"
server_key_iv: "5444d236ad93ff0d4afc55725266ec3c"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:10.950133 19661 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260923 15:08:10.966012 19678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:10.966230 19661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260923 15:08:10.966354 19661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "5207943237114e71b7375daa1dc3fe5e"
format_stamp: "Formatted at 2026-09-23 15:08:10 on dist-test-slave-bb3x"
server_key: "0750dfe173edde643aa29de588d3a0b7"
server_key_iv: "5444d236ad93ff0d4afc55725266ec3c"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:10.966493 19661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:10.988124 19661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:10.990309 19661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:10.990495 19661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:10.990763 19661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:10.991134 19661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:10.991230 19661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:10.991314 19661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:10.991374 19661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:11.000695 19661 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:33035
I20260923 15:08:11.001083 19661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:08:11.003093 19791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:33035 every 8 connection(s)
I20260923 15:08:11.009392 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 19661
I20260923 15:08:11.009536 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:08:11.009758 8169 external_mini_cluster.cc:1428] Setting key 2d7af5cb59c7f44e1088b7cfa2f98a9d
Sep 23 15:08:11 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:11.018157 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.006926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:39589 (local address 127.7.250.126:42121)
0923 15:08:11.006978 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.006985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.009787 (+ 2802us) server_negotiation.cc:407] Connection header received
0923 15:08:11.010596 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.010597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.010612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.010681 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:11.011337 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.011834 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.012841 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.013057 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.013161 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.015257 (+ 2096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:11.015259 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:11.015259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:11.015267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:11.016783 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:11.017185 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:11.017186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:11.017187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:11.017247 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:11.017506 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:11.017507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:11.017508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:11.017625 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:11.017702 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.017974 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.018064 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:08:11.018391 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.006940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42121 (local address 127.7.250.66:39589)
0923 15:08:11.009627 (+ 2687us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.009684 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.010445 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.010772 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.010776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.011038 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:11.011239 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.011243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.011937 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.011938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.012735 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.012737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.013172 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.013173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.013304 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.013835 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:11.013844 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:11.015161 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:11.016875 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:11.016881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:11.016884 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:11.017108 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:11.017314 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:11.017315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:11.017316 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:11.017437 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:11.017693 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:11.017699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:11.017842 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.017953 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.018098 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2637,"thread_start_us":96,"threads_started":1}
I20260923 15:08:11.019039 19792 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42121
I20260923 15:08:11.019162 19792 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:11.019346 19792 heartbeater.cc:511] Master 127.7.250.126:42121 requested a full tablet report, sending...
I20260923 15:08:11.019727 19442 ts_manager.cc:194] Registered new tserver with Master: 5207943237114e71b7375daa1dc3fe5e (127.7.250.66:33035)
I20260923 15:08:11.020105 19442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:39589
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:08:11.042140 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:49681/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42121
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:11.153200 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:11.153586 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:11.153689 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:11.155747 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:11.155870 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:11.156014 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:08:11.158371 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.250.84:49681/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42121
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:11.158655 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:11.158916 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:11.164561 19806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:11.164563 19804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:11.164887 19803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:11.165642 19798 server_base.cc:1034] running on GCE node
I20260923 15:08:11.165948 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:11.166245 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:11.167445 19798 hybrid_clock.cc:648] HybridClock initialized: now 1790176091167421 us; error 41 us; skew 500 ppm
Sep 23 15:08:11 dist-test-slave-bb3x krb5kdc[18957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:11.170549 19798 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:08:11.171123 19798 webserver.cc:460] Webserver started at http://127.7.250.67:34509/ using document root <none> and password file <none>
I20260923 15:08:11.171388 19798 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:11.171460 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:11.171600 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:11 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:11.183596 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "28c38513e0e44c72ab4a1571d7502838"
format_stamp: "Formatted at 2026-09-23 15:08:11 on dist-test-slave-bb3x"
server_key: "d2acc9526841c6dd53751de18b54bfa8"
server_key_iv: "7cb23c82c402eb2de0464034e5b256e8"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:11.184165 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "28c38513e0e44c72ab4a1571d7502838"
format_stamp: "Formatted at 2026-09-23 15:08:11 on dist-test-slave-bb3x"
server_key: "d2acc9526841c6dd53751de18b54bfa8"
server_key_iv: "7cb23c82c402eb2de0464034e5b256e8"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:11.189572 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.001s
I20260923 15:08:11.219539 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:11.219837 19798 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260923 15:08:11.219939 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "28c38513e0e44c72ab4a1571d7502838"
format_stamp: "Formatted at 2026-09-23 15:08:11 on dist-test-slave-bb3x"
server_key: "d2acc9526841c6dd53751de18b54bfa8"
server_key_iv: "7cb23c82c402eb2de0464034e5b256e8"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:11.220054 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:11.232126 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:11.235047 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:11.235234 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:11.235458 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:11.235898 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:11.235949 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:11.235992 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:11.236065 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:11.246233 19798 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:46449
I20260923 15:08:11.254813 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:08:11.258718 19928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:46449 every 8 connection(s)
I20260923 15:08:11.264415 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 19798
I20260923 15:08:11.264680 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:08:11.264904 8169 external_mini_cluster.cc:1428] Setting key f886e378426becf7795f37cba17e9582
Sep 23 15:08:11 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176091, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:11.271353 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.260388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:39191 (local address 127.7.250.126:42121)
0923 15:08:11.260436 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.260438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.260542 (+ 104us) server_negotiation.cc:407] Connection header received
0923 15:08:11.261206 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.261208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.261223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.261316 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:11.262001 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.262933 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.264121 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.264491 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.264590 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.267859 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:11.267861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:11.267861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:11.267871 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:11.269600 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:11.270086 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:11.270087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:11.270088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:11.270188 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:11.270514 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:11.270515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:11.270516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:11.270696 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:11.270777 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.271129 (+ 352us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.271239 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:08:11.271500 19931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.260044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42121 (local address 127.7.250.67:39191)
0923 15:08:11.260272 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.260325 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.261081 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.261373 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.261377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.261646 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:11.261896 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.261901 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.263064 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.263065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.264022 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.264025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.264672 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.264674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.264848 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.265760 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:11.265773 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:11.267755 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:11.269704 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:11.269714 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:11.269717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:11.269978 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:11.270317 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:11.270317 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:11.270318 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:11.270445 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:11.270771 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:11.270776 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:11.270986 (+ 210us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.271080 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.271209 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":120,"threads_started":1}
I20260923 15:08:11.272096 19929 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42121
I20260923 15:08:11.272223 19929 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:11.272444 19929 heartbeater.cc:511] Master 127.7.250.126:42121 requested a full tablet report, sending...
I20260923 15:08:11.273046 19442 ts_manager.cc:194] Registered new tserver with Master: 28c38513e0e44c72ab4a1571d7502838 (127.7.250.67:46449)
I20260923 15:08:11.273518 19442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:39191
I20260923 15:08:11.277173 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:08:11.287387 19657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.279044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53468 (local address 127.7.250.126:42121)
0923 15:08:11.279137 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.279140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.279349 (+ 209us) server_negotiation.cc:407] Connection header received
0923 15:08:11.279407 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.279408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.279424 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.279513 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:11.280134 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.281275 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.282069 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.282535 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.282650 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.284074 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:11.284077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:11.284077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:11.284087 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:11.285816 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:11.286284 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:11.286285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:11.286286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:11.286349 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:11.286569 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:11.286570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:11.286570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:11.286701 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:11.286792 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.287214 (+ 422us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.287292 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:08:11.291229 19442 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:08:11.293748 19442 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:08:11.308758 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.301462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33035 (local address 127.0.0.1:37568)
0923 15:08:11.303127 (+ 1665us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.303137 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.303176 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.303846 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.303851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.303854 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:08:11.303943 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.303946 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.305609 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.305610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.307165 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.307169 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.308416 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.308417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.308518 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.308534 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.308608 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.308661 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1621,"thread_start_us":137,"threads_started":1}
I20260923 15:08:11.309394 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.302035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.7.250.66:33035)
0923 15:08:11.303575 (+ 1540us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.303581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.303592 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:08:11.303658 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.303661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.303679 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.303760 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:08:11.304062 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.305239 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.307334 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.308221 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.308297 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.308379 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.309261 (+ 882us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.309305 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1477,"thread_start_us":116,"threads_started":1}
I20260923 15:08:11.310213 19726 tablet_service.cc:1467] Processing CreateTablet for tablet 6421898c598d4443b4b5d66831df5f07 (DEFAULT_TABLE table=test-table [id=aa45c52bb8544028a097c587f5bb80e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:08:11.310637 19726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6421898c598d4443b4b5d66831df5f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:08:11.312011 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.301173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:46449 (local address 127.0.0.1:38424)
0923 15:08:11.302813 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.302861 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.302931 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.304798 (+ 1867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.304800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.304802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:08:11.304855 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.304857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.307349 (+ 2492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.307349 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.308669 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.308672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.311751 (+ 3079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.311752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.311825 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.311829 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.311883 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.311927 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1581,"thread_start_us":101,"threads_started":1}
I20260923 15:08:11.312340 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.301287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38424 (local address 127.7.250.67:46449)
0923 15:08:11.304059 (+ 2772us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.304064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.304075 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:08:11.304138 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.304141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.304158 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.304605 (+ 447us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:08:11.304949 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.306474 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.308898 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.311560 (+ 2662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.311637 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.311714 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.312189 (+ 475us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.312240 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2706,"thread_start_us":78,"threads_started":1}
I20260923 15:08:11.313046 19861 tablet_service.cc:1467] Processing CreateTablet for tablet 6421898c598d4443b4b5d66831df5f07 (DEFAULT_TABLE table=test-table [id=aa45c52bb8544028a097c587f5bb80e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:08:11.313411 19861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6421898c598d4443b4b5d66831df5f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:08:11.318456 19948 tablet_bootstrap.cc:492] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Bootstrap starting.
I20260923 15:08:11.320355 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.301697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37523 (local address 127.0.0.1:37370)
0923 15:08:11.303321 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.303331 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.303372 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.307128 (+ 3756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.307129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.307131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:08:11.307230 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.307232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.317432 (+ 10200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.317433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.318781 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.318785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.320101 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.320102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.320184 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.320190 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.320225 (+ 35us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.320274 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1578,"thread_start_us":140,"threads_started":1}
I20260923 15:08:11.320717 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.301800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37370 (local address 127.7.250.65:37523)
0923 15:08:11.306907 (+ 5107us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.306912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.306922 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:08:11.306957 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.306959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.306977 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.307057 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:08:11.309642 (+ 2585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.310882 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.318941 (+ 8059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.319938 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.320480 (+ 542us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.320568 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.320600 (+ 32us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.320647 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5056,"thread_start_us":85,"threads_started":1}
I20260923 15:08:11.321038 19949 tablet_bootstrap.cc:492] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Bootstrap starting.
I20260923 15:08:11.321488 19589 tablet_service.cc:1467] Processing CreateTablet for tablet 6421898c598d4443b4b5d66831df5f07 (DEFAULT_TABLE table=test-table [id=aa45c52bb8544028a097c587f5bb80e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:08:11.321882 19589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6421898c598d4443b4b5d66831df5f07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:08:11.322597 19949 tablet_bootstrap.cc:654] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Neither blocks nor log segments found. Creating new log.
I20260923 15:08:11.322966 19949 log.cc:824] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Log is configured to *not* fsync() on all Append() calls
I20260923 15:08:11.323760 19949 tablet_bootstrap.cc:492] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: No bootstrap required, opened a new log
I20260923 15:08:11.323860 19949 ts_tablet_manager.cc:1397] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:08:11.325263 19948 tablet_bootstrap.cc:654] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Neither blocks nor log segments found. Creating new log.
I20260923 15:08:11.325613 19948 log.cc:824] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:08:11.326040 19951 tablet_bootstrap.cc:492] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd: Bootstrap starting.
I20260923 15:08:11.325948 19949 raft_consensus.cc:348] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.326112 19949 raft_consensus.cc:374] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:08:11.326169 19949 raft_consensus.cc:729] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28c38513e0e44c72ab4a1571d7502838, State: Initialized, Role: FOLLOWER
I20260923 15:08:11.326287 19949 consensus_queue.cc:260] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.326589 19949 ts_tablet_manager.cc:1428] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:08:11.326769 19929 heartbeater.cc:503] Master 127.7.250.126:42121 was elected leader, sending a full tablet report...
I20260923 15:08:11.328599 19951 tablet_bootstrap.cc:654] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd: Neither blocks nor log segments found. Creating new log.
I20260923 15:08:11.328739 19948 tablet_bootstrap.cc:492] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: No bootstrap required, opened a new log
I20260923 15:08:11.328809 19948 ts_tablet_manager.cc:1397] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260923 15:08:11.329624 19951 log.cc:824] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:08:11.330426 19951 tablet_bootstrap.cc:492] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd: No bootstrap required, opened a new log
I20260923 15:08:11.330492 19951 ts_tablet_manager.cc:1397] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260923 15:08:11.330875 19948 raft_consensus.cc:348] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.331045 19948 raft_consensus.cc:374] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:08:11.331079 19948 raft_consensus.cc:729] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5207943237114e71b7375daa1dc3fe5e, State: Initialized, Role: FOLLOWER
I20260923 15:08:11.331183 19948 consensus_queue.cc:260] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.331418 19948 ts_tablet_manager.cc:1428] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:08:11.331532 19792 heartbeater.cc:503] Master 127.7.250.126:42121 was elected leader, sending a full tablet report...
I20260923 15:08:11.333026 19951 raft_consensus.cc:348] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.333235 19951 raft_consensus.cc:374] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:08:11.333357 19951 raft_consensus.cc:729] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b07303d36b4430299f6e86bf5c35dcd, State: Initialized, Role: FOLLOWER
I20260923 15:08:11.333521 19951 consensus_queue.cc:260] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.333775 19951 ts_tablet_manager.cc:1428] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:08:11.333803 19655 heartbeater.cc:503] Master 127.7.250.126:42121 was elected leader, sending a full tablet report...
I20260923 15:08:11.421907 19956 raft_consensus.cc:482] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:08:11.422091 19956 raft_consensus.cc:504] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.422983 19956 leader_election.cc:288] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28c38513e0e44c72ab4a1571d7502838 (127.7.250.67:46449), 5207943237114e71b7375daa1dc3fe5e (127.7.250.66:33035)
I20260923 15:08:11.427299 19943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.422895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:45941 (local address 127.7.250.67:46449)
0923 15:08:11.422961 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.422963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.423198 (+ 235us) server_negotiation.cc:407] Connection header received
0923 15:08:11.423299 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.423300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.423325 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.423376 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:08:11.423900 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.424719 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.426343 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.426912 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.426960 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.426969 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.427186 (+ 217us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.427228 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:08:11.427526 19958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.422671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:46449 (local address 127.7.250.65:45941)
0923 15:08:11.423147 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.423155 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.423226 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.423395 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.423396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.423398 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:08:11.423663 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.423667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.424880 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.424881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.426217 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.426220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.427028 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.427029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.427100 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.427104 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.427402 (+ 298us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.427444 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":127,"threads_started":1}
I20260923 15:08:11.427887 19863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6421898c598d4443b4b5d66831df5f07" candidate_uuid: "4b07303d36b4430299f6e86bf5c35dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c38513e0e44c72ab4a1571d7502838" is_pre_election: true
I20260923 15:08:11.427984 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.422875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:55589 (local address 127.7.250.66:33035)
0923 15:08:11.422922 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.422924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.423058 (+ 134us) server_negotiation.cc:407] Connection header received
0923 15:08:11.423526 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.423527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.423537 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.423570 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:08:11.423803 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.424874 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.426426 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.427150 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.427196 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.427203 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.427862 (+ 659us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.427904 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:08:11.428076 19863 raft_consensus.cc:2454] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b07303d36b4430299f6e86bf5c35dcd in term 0.
I20260923 15:08:11.428304 19542 leader_election.cc:302] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28c38513e0e44c72ab4a1571d7502838, 4b07303d36b4430299f6e86bf5c35dcd; no voters:
I20260923 15:08:11.428445 19956 raft_consensus.cc:2781] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:08:11.428517 19956 raft_consensus.cc:482] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:08:11.428545 19956 raft_consensus.cc:3037] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:08:11.428615 19957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.422667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33035 (local address 127.7.250.65:55589)
0923 15:08:11.422988 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.422999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:11.423282 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:11.423642 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:11.423643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:11.423644 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:08:11.423725 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.423727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.425076 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.425077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.426332 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:11.426334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.427262 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.427263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:11.427343 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.427346 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:08:11.428289 (+ 943us) client_negotiation.cc:240] Negotiation successful
0923 15:08:11.428335 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":15,"thread_start_us":95,"threads_started":1}
I20260923 15:08:11.429061 19746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6421898c598d4443b4b5d66831df5f07" candidate_uuid: "4b07303d36b4430299f6e86bf5c35dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5207943237114e71b7375daa1dc3fe5e" is_pre_election: true
I20260923 15:08:11.429194 19746 raft_consensus.cc:2454] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b07303d36b4430299f6e86bf5c35dcd in term 0.
I20260923 15:08:11.430032 19956 raft_consensus.cc:504] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.430167 19956 leader_election.cc:288] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [CANDIDATE]: Term 1 election: Requested vote from peers 28c38513e0e44c72ab4a1571d7502838 (127.7.250.67:46449), 5207943237114e71b7375daa1dc3fe5e (127.7.250.66:33035)
I20260923 15:08:11.430373 19863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6421898c598d4443b4b5d66831df5f07" candidate_uuid: "4b07303d36b4430299f6e86bf5c35dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28c38513e0e44c72ab4a1571d7502838"
I20260923 15:08:11.430377 19746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6421898c598d4443b4b5d66831df5f07" candidate_uuid: "4b07303d36b4430299f6e86bf5c35dcd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5207943237114e71b7375daa1dc3fe5e"
I20260923 15:08:11.430449 19746 raft_consensus.cc:3037] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:08:11.430449 19863 raft_consensus.cc:3037] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:08:11.431273 19746 raft_consensus.cc:2454] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b07303d36b4430299f6e86bf5c35dcd in term 1.
I20260923 15:08:11.431519 19544 leader_election.cc:302] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b07303d36b4430299f6e86bf5c35dcd, 5207943237114e71b7375daa1dc3fe5e; no voters:
I20260923 15:08:11.431654 19956 raft_consensus.cc:2781] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:08:11.431731 19863 raft_consensus.cc:2454] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b07303d36b4430299f6e86bf5c35dcd in term 1.
I20260923 15:08:11.431867 19956 raft_consensus.cc:686] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [term 1 LEADER]: Becoming Leader. State: Replica: 4b07303d36b4430299f6e86bf5c35dcd, State: Running, Role: LEADER
I20260923 15:08:11.432005 19956 consensus_queue.cc:237] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } }
I20260923 15:08:11.432833 19442 catalog_manager.cc:5495] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b07303d36b4430299f6e86bf5c35dcd (127.7.250.65). New cstate: current_term: 1 leader_uuid: "4b07303d36b4430299f6e86bf5c35dcd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b07303d36b4430299f6e86bf5c35dcd" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37523 } health_report { overall_health: HEALTHY } } }
Sep 23 15:08:11 dist-test-slave-bb3x krb5kdc[18957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176090, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:11.456748 19946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.452846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37386 (local address 127.7.250.65:37523)
0923 15:08:11.452908 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:11.452911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:11.452920 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:08:11.452968 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:11.452968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:11.452986 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:11.453084 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:08:11.453233 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.454035 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.455328 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:11.455652 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:11.455726 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:11.455976 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:08:11.456141 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:08:11.456225 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:11.456484 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 15:08:11.456509 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:08:11.459151 19746 raft_consensus.cc:1261] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 1 FOLLOWER]: Refusing update from remote peer 4b07303d36b4430299f6e86bf5c35dcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:08:11.459486 19959 consensus_queue.cc:1035] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5207943237114e71b7375daa1dc3fe5e" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:08:11.461583 19863 raft_consensus.cc:1261] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 1 FOLLOWER]: Refusing update from remote peer 4b07303d36b4430299f6e86bf5c35dcd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:08:11.461854 19959 consensus_queue.cc:1035] T 6421898c598d4443b4b5d66831df5f07 P 4b07303d36b4430299f6e86bf5c35dcd [LEADER]: Connected to new peer: Peer: permanent_uuid: "28c38513e0e44c72ab4a1571d7502838" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 46449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:08:11.466503 19962 mvcc.cc:205] Tried to move back new op lower bound from 7332561270613995520 to 7332561270506262528. Current Snapshot: MvccSnapshot[applied={T|T < 7332561270613995520}]
I20260923 15:08:11.472036 19965 mvcc.cc:205] Tried to move back new op lower bound from 7332561270613995520 to 7332561270506262528. Current Snapshot: MvccSnapshot[applied={T|T < 7332561270613995520}]
I20260923 15:08:11.472705 19441 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:08:11.472815 19441 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:08:11.473701 19441 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64004eb4b94545d2b7da5db46e47e85b: Sending DeleteTablet for 3 replicas of tablet 6421898c598d4443b4b5d66831df5f07
I20260923 15:08:11.474156 19861 tablet_service.cc:1514] Processing DeleteTablet for tablet 6421898c598d4443b4b5d66831df5f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:08:11 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:38424
I20260923 15:08:11.474422 19726 tablet_service.cc:1514] Processing DeleteTablet for tablet 6421898c598d4443b4b5d66831df5f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:08:11 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37568
I20260923 15:08:11.474481 19589 tablet_service.cc:1514] Processing DeleteTablet for tablet 6421898c598d4443b4b5d66831df5f07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:08:11 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37370
I20260923 15:08:11.474706 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 19524
I20260923 15:08:11.477020 19972 tablet_replica.cc:330] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: stopping tablet replica
I20260923 15:08:11.477192 19972 raft_consensus.cc:2229] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:08:11.477416 19972 raft_consensus.cc:2258] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:08:11.477715 19971 tablet_replica.cc:330] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: stopping tablet replica
I20260923 15:08:11.477878 19971 raft_consensus.cc:2229] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:08:11.478046 19971 raft_consensus.cc:2258] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:08:11.478636 19971 ts_tablet_manager.cc:1905] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:08:11.480574 19971 ts_tablet_manager.cc:1918] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:08:11.481117 19972 ts_tablet_manager.cc:1905] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:08:11.481254 19971 log.cc:1196] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/6421898c598d4443b4b5d66831df5f07
I20260923 15:08:11.481966 19971 ts_tablet_manager.cc:1939] T 6421898c598d4443b4b5d66831df5f07 P 28c38513e0e44c72ab4a1571d7502838: Deleting consensus metadata
I20260923 15:08:11.483748 19972 ts_tablet_manager.cc:1918] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:08:11.483837 19972 log.cc:1196] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/6421898c598d4443b4b5d66831df5f07
I20260923 15:08:11.484130 19972 ts_tablet_manager.cc:1939] T 6421898c598d4443b4b5d66831df5f07 P 5207943237114e71b7375daa1dc3fe5e: Deleting consensus metadata
I20260923 15:08:11.484561 19428 catalog_manager.cc:4841] TS 5207943237114e71b7375daa1dc3fe5e (127.7.250.66:33035): tablet 6421898c598d4443b4b5d66831df5f07 (table test-table [id=aa45c52bb8544028a097c587f5bb80e2]) successfully deleted
I20260923 15:08:11.484503 19426 catalog_manager.cc:4841] TS 28c38513e0e44c72ab4a1571d7502838 (127.7.250.67:46449): tablet 6421898c598d4443b4b5d66831df5f07 (table test-table [id=aa45c52bb8544028a097c587f5bb80e2]) successfully deleted
W20260923 15:08:11.485062 19428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:37523 (error 108)
I20260923 15:08:11.485337 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 19661
I20260923 15:08:11.486729 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:11.486599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37523 (local address 127.0.0.1:37390)
0923 15:08:11.486653 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:11.486692 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:37523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:08:11.486863 19428 catalog_manager.cc:4584] TS 4b07303d36b4430299f6e86bf5c35dcd (127.7.250.65:37523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6421898c598d4443b4b5d66831df5f07: Network error: Client connection negotiation failed: client connection to 127.7.250.65:37523: connect: Connection refused (error 111)
I20260923 15:08:11.491379 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 19798
I20260923 15:08:11.497592 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 19410
I20260923 15:08:11.507859 8169 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 15:08:11.852440 8169 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 15:08:11.852567 8169 mini_ranger.cc:66] Stopping Ranger...
I20260923 15:08:12.207741 8169 mini_ranger.cc:68] Stopped Ranger
2026-09-23 15:08:12.207 UTC [18988] LOG: received smart shutdown request
2026-09-23 15:08:12.212 UTC [18988] LOG: background worker "logical replication launcher" (PID 18997) exited with exit code 1
2026-09-23 15:08:12.213 UTC [18992] LOG: shutting down
2026-09-23 15:08:12.411 UTC [18988] LOG: database system is shut down
2026-09-23T15:08:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52482 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:08:12 dist-test-slave-bb3x krb5kdc[19983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:08:12 dist-test-slave-bb3x krb5kdc[19983](info): set up 2 sockets
Sep 23 15:08:12 dist-test-slave-bb3x krb5kdc[19983](info): commencing operation
krb5kdc: starting...
W20260923 15:08:13.568033 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:08:13 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176093, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:08:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/rangeradmin_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/rangeradmin_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/rangerlookup_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/rangerlookup_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
I20260923 15:08:13.741817 8169 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskOOn7ma/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/postgres -l logfile start
2026-09-23 15:08:16.488 UTC [20014] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 15:08:16.488 UTC [20014] LOG: listening on IPv4 address "127.7.250.84", port 46381
2026-09-23 15:08:16.492 UTC [20014] LOG: listening on Unix socket "/tmp/.s.PGSQL.46381"
2026-09-23 15:08:16.514 UTC [20017] LOG: database system was shut down at 2026-09-23 15:08:14 UTC
2026-09-23 15:08:16.518 UTC [20014] LOG: database system is ready to accept connections
I20260923 15:08:17.472311 8169 mini_postgres.cc:96] Postgres bound to 46381
2026-09-23 15:08:17.479 UTC [20025] FATAL: database "slave" does not exist
127.7.250.84:46381 - accepting connections
I20260923 15:08:17.479753 8169 mini_ranger.cc:161] Starting Ranger...
I20260923 15:08:17.495082 8169 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 15:08:18.015116 8169 mini_ranger.cc:87] Created ranger Postgres database
I20260923 15:08:18.015190 8169 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-admin
2026-09-23 15:08:18,307 [I] DB FLAVOR :POSTGRES
2026-09-23 15:08:18,308 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:08:18,308 [I] Checking connection..
2026-09-23 15:08:18,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 15:08:18,541 [I] Checking connection passed.
2026-09-23 15:08:18,541 [I] --------- Verifying version history table ---------
2026-09-23 15:08:18,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:08:18,790 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:08:18,790 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:08:19,057 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:08:19,057 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 15:08:19,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 15:08:19.294 UTC [20089] WARNING: there is no transaction in progress
2026-09-23 15:08:19,301 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 15:08:19,302 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:08:19,547 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 15:08:19,547 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:08:19,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 15:08:19,785 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 15:08:20,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bb3x.c.gcp-upstream.internal','N') ;"
2026-09-23 15:08:20,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:08:20,560 [I] Table x_portal_user does not exist in database ranger
2026-09-23 15:08:20,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:08:20,801 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 15:08:20,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:08:21,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:08:21,294 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 15:08:21,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 15:08:22.314 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.332 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.351 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.368 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.385 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.406 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.464 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.472 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.487 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.498 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.509 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.525 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.535 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.542 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.549 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:22.556 UTC [20198] WARNING: there is no transaction in progress
2026-09-23 15:08:23,309 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 15:08:23,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 15:08:23,550 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 15:08:23,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:08:23,787 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 15:08:23,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:08:24,021 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 15:08:24,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:08:24,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:08:24,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bb3x.c.gcp-upstream.internal';"
2026-09-23 15:08:24,747 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 15:08:24,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 15:08:25,000 [I] DB_PATCHES have already been applied
I20260923 15:08:25.005843 8169 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-admin:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskOOn7ma/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 15:08:25.005911 8169 mini_ranger.cc:193] Using host: 127.7.250.84
I20260923 15:08:25.009014 8169 mini_ranger.cc:227] Ranger admin URL: http://127.7.250.84:53391
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 3:08:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 15:08:26.109745 8169 mini_ranger.cc:160] Time spent starting Ranger: real 8.630s user 0.000s sys 0.009s
Sep 23, 2026 3:08:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 3:08:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 3:08:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 3:08:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 3:08:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 15:08:47.803817 8169 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.250.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.250.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/rangerkms_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.250.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/rangerkms_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.250.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.250.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.250.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/HTTP_127.7.250.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 15:08:47 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176127, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 15:08:47.893949 8169 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 15:08:47.904036 8169 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 15:08:48.982112 8169 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 15:08:48.982219 8169 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms
I20260923 15:08:48.982240 8169 mini_ranger_kms.cc:227] Using postgres at 127.7.250.84:46381
2026-09-23 15:08:49,154 [I] DB FLAVOR :POSTGRES
2026-09-23 15:08:49,154 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:08:49,154 [I] Checking connection
2026-09-23 15:08:49,154 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 15:08:49,396 [I] connection success
2026-09-23 15:08:49,397 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 15:08:49,397 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 15:08:49,397 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 15:08:49,679 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 15:08:49,680 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:08:49,680 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 15:08:49,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.250.84:46381/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 15:08:49,958 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 15:08:50.506585 8169 mini_ranger_kms.cc:310] Created kms service
I20260923 15:08:50.707145 8169 mini_ranger_kms.cc:326] Created kudu user
I20260923 15:08:50.753500 8169 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 15:08:50 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176127, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:51.153968 8169 mini_ranger_kms.cc:384] Added ranger policy
I20260923 15:08:51.154177 8169 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms:/tmp/dist-test-taskOOn7ma/build/release/bin/postgresql.jar:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskOOn7ma/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 15:08:51.154201 8169 mini_ranger_kms.cc:240] Using host: 127.7.250.84
I20260923 15:08:51.157855 8169 mini_ranger_kms.cc:276] Ranger KMS PID: 20380
I20260923 15:08:51.157953 8169 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.250.84:48875
Sep 23, 2026 3:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 3:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 3:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 3:08:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 15:08:53.521870 8169 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.628s user 0.007s sys 0.148s
I20260923 15:08:53.521986 8169 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 15:08:53.522042 8169 mini_ranger_kms.cc:401] 127.7.250.84:48875/kms/v1/keys
Sep 23 15:08:53 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790176133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.250.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 15:08:54 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.250.84@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
Sep 23 15:08:54 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176133, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.250.84@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
Sep 23 15:08:56 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:08:56.853027 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37969
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:37969
--ranger_config_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:56.943097 20431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:56.943322 20431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:56.943347 20431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:56.944571 20431 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:08:56.944617 20431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:56.944631 20431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:56.944644 20431 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:08:56.944656 20431 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:08:56.946436 20431 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.250.84:48875/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:37969
--ranger_config_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:56.946700 20431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:56.946973 20431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:56.952701 20436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:56.952811 20431 server_base.cc:1034] running on GCE node
W20260923 15:08:56.952714 20437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:56.952761 20440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:56.953217 20431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:56.953549 20431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:56.954715 20431 hybrid_clock.cc:648] HybridClock initialized: now 1790176136954690 us; error 41 us; skew 500 ppm
Sep 23 15:08:56 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:56.965807 20431 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:08:56.966461 20431 webserver.cc:460] Webserver started at http://127.7.250.126:39701/ using document root <none> and password file <none>
I20260923 15:08:56.966743 20431 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:56.966787 20431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:56.966940 20431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:56 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176136, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:57.023272 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "25f82cee2ac147c7b62e832b2bcdb5d9"
format_stamp: "Formatted at 2026-09-23 15:08:57 on dist-test-slave-bb3x"
server_key: "69fa3f0c1b1de6c38fce72f9bd15f655"
server_key_iv: "72136963e0b49da433664a43e8ee7b4d"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:57.023734 20431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "25f82cee2ac147c7b62e832b2bcdb5d9"
format_stamp: "Formatted at 2026-09-23 15:08:57 on dist-test-slave-bb3x"
server_key: "69fa3f0c1b1de6c38fce72f9bd15f655"
server_key_iv: "72136963e0b49da433664a43e8ee7b4d"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:57.025614 20431 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260923 15:08:57.040182 20447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:57.040462 20431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:08:57.040575 20431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "25f82cee2ac147c7b62e832b2bcdb5d9"
format_stamp: "Formatted at 2026-09-23 15:08:57 on dist-test-slave-bb3x"
server_key: "69fa3f0c1b1de6c38fce72f9bd15f655"
server_key_iv: "72136963e0b49da433664a43e8ee7b4d"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:57.040705 20431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:57.051637 20431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:57.053864 20431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:57.054028 20431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:57.058984 20431 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:37969
I20260923 15:08:57.059013 20499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:37969 every 8 connection(s)
I20260923 15:08:57.059413 20431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:08:57.060114 20500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:08:57.062613 20500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Bootstrap starting.
I20260923 15:08:57.063248 20500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Neither blocks nor log segments found. Creating new log.
I20260923 15:08:57.063555 20500 log.cc:824] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:08:57.064360 20500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: No bootstrap required, opened a new log
I20260923 15:08:57.066551 20500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:57.066712 20500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:08:57.066737 20500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f82cee2ac147c7b62e832b2bcdb5d9, State: Initialized, Role: FOLLOWER
I20260923 15:08:57.066900 20500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:57.066978 20500 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:08:57.067022 20500 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:08:57.067071 20500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:08:57.067651 20500 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:57.067739 20500 leader_election.cc:302] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25f82cee2ac147c7b62e832b2bcdb5d9; no voters:
I20260923 15:08:57.067960 20500 leader_election.cc:288] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:08:57.067989 20503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:08:57.068101 20503 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 LEADER]: Becoming Leader. State: Replica: 25f82cee2ac147c7b62e832b2bcdb5d9, State: Running, Role: LEADER
I20260923 15:08:57.068190 20503 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:57.068251 20500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:08:57.068537 20503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } } }
I20260923 15:08:57.068686 20503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:08:57.068877 20503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25f82cee2ac147c7b62e832b2bcdb5d9. Latest consensus state: current_term: 1 leader_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } } }
I20260923 15:08:57.068980 20503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:08:57.069279 20509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:08:57.069502 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 20431
I20260923 15:08:57.069614 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:08:57.069833 8169 external_mini_cluster.cc:1428] Setting key 43d015263137cce9a5e458d3973fdc7f
I20260923 15:08:57.069861 20509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:08:57.070641 20500 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 15:08:57.075839 20509 catalog_manager.cc:1347] Generated new cluster ID: f09286c42b7c4941b5767518a73c81ff
I20260923 15:08:57.075997 20509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:08:57.087604 20509 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:08:57.088398 20509 catalog_manager.cc:1504] Loading token signing keys...
Sep 23 15:08:57 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:57.103735 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:57.070416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.7.250.126:37969)
0923 15:08:57.074474 (+ 4058us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:57.074485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:57.074551 (+ 66us) server_negotiation.cc:407] Connection header received
0923 15:08:57.074819 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:57.074831 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:57.074918 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:57.076001 (+ 1083us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:57.076144 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:57.077557 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:57.087484 (+ 9927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:57.088139 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:57.088613 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:57.099391 (+ 10778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:57.099397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:57.099398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:57.099416 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:57.101447 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:57.101885 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:57.101889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:57.101889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:57.102006 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:57.102225 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:57.102226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:57.102226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:57.102557 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:57.102766 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:57.103322 (+ 556us) server_negotiation.cc:299] Negotiation successful
0923 15:08:57.103479 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3949,"thread_start_us":73,"threads_started":1}
I20260923 15:08:57.115582 20509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Generated new TSK 0
I20260923 15:08:57.115897 20509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:08:57.878180 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37969
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:58.004170 20545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:58.004468 20545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:58.004508 20545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:58.006342 20545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:58.006418 20545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:58.006459 20545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:08:58.008808 20545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.250.84:48875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:58.009308 20545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:58.009620 20545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:58.016454 20551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:58.016785 20553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:58.018025 20545 server_base.cc:1034] running on GCE node
W20260923 15:08:58.018191 20550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:58.018466 20545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:58.018738 20545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:58.019922 20545 hybrid_clock.cc:648] HybridClock initialized: now 1790176138019898 us; error 32 us; skew 500 ppm
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:58.023185 20545 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:08:58.023701 20545 webserver.cc:460] Webserver started at http://127.7.250.65:34941/ using document root <none> and password file <none>
I20260923 15:08:58.023998 20545 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:58.024058 20545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:58.024183 20545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:58.039975 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "e93cf8c10c6d892395979e270986d458"
server_key_iv: "dac80c0f82c122c0a664f3c227979522"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.040542 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "e93cf8c10c6d892395979e270986d458"
server_key_iv: "dac80c0f82c122c0a664f3c227979522"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.042441 20545 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 15:08:58.057205 20561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.057448 20545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:08:58.057530 20545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "e93cf8c10c6d892395979e270986d458"
server_key_iv: "dac80c0f82c122c0a664f3c227979522"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.057616 20545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:58.086920 20545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:58.089402 20545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:58.089589 20545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:58.089870 20545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:58.090349 20545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:58.090402 20545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.090436 20545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:58.090459 20545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.097035 20545 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43357
I20260923 15:08:58.097566 20545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:08:58.101430 20676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43357 every 8 connection(s)
I20260923 15:08:58.105414 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 20545
I20260923 15:08:58.105530 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:08:58.105712 8169 external_mini_cluster.cc:1428] Setting key c316d2eb2647a309bfbdb40d23acfe72
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:58.116580 20679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:58.102136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37969 (local address 127.7.250.65:34145)
0923 15:08:58.102769 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:58.102815 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:58.103407 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:58.103904 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:58.103908 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:58.104188 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:58.104479 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:58.104486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.108233 (+ 3747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.108234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:58.109467 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:58.109470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.110056 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.110057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:58.110201 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:58.110859 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:58.110869 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:58.112477 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:58.114724 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:58.114732 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:58.114735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:58.115031 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:58.115290 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:58.115291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:58.115292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:58.115426 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:58.115789 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:58.115797 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:58.115975 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 15:08:58.116101 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 15:08:58.116278 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":120,"threads_started":1}
I20260923 15:08:58.117233 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:58.102469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:34145 (local address 127.7.250.126:37969)
0923 15:08:58.103606 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:58.103608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:58.103617 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:08:58.103644 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:58.103645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:58.103658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:58.103718 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:58.104603 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.108123 (+ 3520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.109587 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.109900 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.109967 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:58.112591 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:58.112593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:58.112594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:58.112603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:58.114625 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:58.115130 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:58.115132 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:58.115133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:58.115207 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:58.115506 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:58.115507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:58.115508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:58.115661 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:58.115767 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:58.116889 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0923 15:08:58.116991 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":84,"threads_started":1}
I20260923 15:08:58.117633 20677 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37969
I20260923 15:08:58.117743 20677 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:58.117939 20677 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:08:58.118551 20464 ts_manager.cc:194] Registered new tserver with Master: 0b6ee3a4ffbb41469c97600e9b5c42ad (127.7.250.65:43357)
I20260923 15:08:58.119470 20464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:34145
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:08:58.149046 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37969
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 15:08:58.290365 20683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:58.290674 20683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:58.290720 20683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:58.292853 20683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:58.292996 20683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:58.293098 20683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:08:58.295892 20683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.250.84:48875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:58.296420 20683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:58.296797 20683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:58.306617 20694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:58.306910 20692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:58.307895 20691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:58.308224 20683 server_base.cc:1034] running on GCE node
I20260923 15:08:58.308470 20683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:58.308758 20683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:58.309942 20683 hybrid_clock.cc:648] HybridClock initialized: now 1790176138309922 us; error 34 us; skew 500 ppm
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:58.313613 20683 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:08:58.314285 20683 webserver.cc:460] Webserver started at http://127.7.250.66:40143/ using document root <none> and password file <none>
I20260923 15:08:58.314631 20683 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:58.314752 20683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:58.314944 20683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:58.331010 20683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "d5ca166b6d3640058a5062f6557638c4"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "66abe58ae260afdc0d0337dba7a66170"
server_key_iv: "3d700224196568898309ad93423ee402"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.331583 20683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "d5ca166b6d3640058a5062f6557638c4"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "66abe58ae260afdc0d0337dba7a66170"
server_key_iv: "3d700224196568898309ad93423ee402"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.338707 20683 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260923 15:08:58.352186 20701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.352466 20683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:08:58.352564 20683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "d5ca166b6d3640058a5062f6557638c4"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "66abe58ae260afdc0d0337dba7a66170"
server_key_iv: "3d700224196568898309ad93423ee402"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.352666 20683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:58.379128 20683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:58.381600 20683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:58.381747 20683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:58.381960 20683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:58.382397 20683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:58.382449 20683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.382480 20683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:58.382529 20683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.389204 20683 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:34841
I20260923 15:08:58.389488 20816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:34841 every 8 connection(s)
I20260923 15:08:58.389715 20683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:08:58.390344 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 20683
I20260923 15:08:58.390465 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:08:58.390689 8169 external_mini_cluster.cc:1428] Setting key 4c81cfa0c84a85f627291df18d8c4b5a
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
I20260923 15:08:58.405817 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:58.391401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:53019 (local address 127.7.250.126:37969)
0923 15:08:58.391508 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:58.391510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:58.394616 (+ 3106us) server_negotiation.cc:407] Connection header received
0923 15:08:58.395411 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:58.395413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:58.395428 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:58.395507 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:58.396230 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.397031 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.398648 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.399017 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.399075 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:58.402236 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:58.402238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:58.402239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:58.402248 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:58.404054 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:58.404613 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:58.404614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:58.404615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:58.404681 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:58.404967 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:58.404968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:58.404968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:58.405095 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:58.405175 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:58.405581 (+ 406us) server_negotiation.cc:299] Negotiation successful
0923 15:08:58.405723 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 15:08:58.405963 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:58.391412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37969 (local address 127.7.250.66:53019)
0923 15:08:58.394456 (+ 3044us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:58.394509 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:58.395291 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:58.395588 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:58.395593 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:58.395840 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:58.396115 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:58.396121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.397154 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.397156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:58.398546 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:58.398549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.399205 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.399206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:58.399374 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:58.400222 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:58.400233 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:58.402093 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:58.404160 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:58.404180 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:58.404182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:58.404514 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:58.404765 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:58.404766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:58.404767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:58.404887 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:58.405250 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:58.405256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:58.405428 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 15:08:58.405548 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 15:08:58.405711 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2991,"thread_start_us":94,"threads_started":1}
I20260923 15:08:58.406551 20817 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37969
I20260923 15:08:58.406673 20817 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:58.406886 20817 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
I20260923 15:08:58.407281 20464 ts_manager.cc:194] Registered new tserver with Master: d5ca166b6d3640058a5062f6557638c4 (127.7.250.66:34841)
I20260923 15:08:58.407784 20464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:53019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:08:58.425376 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37969
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:58.556173 20822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:58.556438 20822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:58.556476 20822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:58.557744 20822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:58.557788 20822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:58.557816 20822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:08:58.559394 20822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.250.84:48875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:58.559648 20822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:58.559914 20822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:58.566499 20830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:58.566566 20828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:58.566839 20822 server_base.cc:1034] running on GCE node
W20260923 15:08:58.566560 20827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:58.567224 20822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:58.567503 20822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:58.568665 20822 hybrid_clock.cc:648] HybridClock initialized: now 1790176138568636 us; error 41 us; skew 500 ppm
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:58.571657 20822 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:08:58.572224 20822 webserver.cc:460] Webserver started at http://127.7.250.67:43815/ using document root <none> and password file <none>
I20260923 15:08:58.572497 20822 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:58.572568 20822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:08:58.572779 20822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:58.593053 20822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "cfcb36195a574b4782990f9d4031f3d8"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "9fb6aee822c6337df558db3029ba8c9c"
server_key_iv: "7d1286a52e1215d963a0e57f2b45f82b"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.593550 20822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "cfcb36195a574b4782990f9d4031f3d8"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "9fb6aee822c6337df558db3029ba8c9c"
server_key_iv: "7d1286a52e1215d963a0e57f2b45f82b"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.598342 20822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260923 15:08:58.620882 20837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.621250 20822 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260923 15:08:58.621428 20822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "cfcb36195a574b4782990f9d4031f3d8"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "9fb6aee822c6337df558db3029ba8c9c"
server_key_iv: "7d1286a52e1215d963a0e57f2b45f82b"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.621557 20822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:58.635293 20822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:58.638432 20822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:58.638679 20822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:58.638995 20822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:58.639432 20822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:58.639523 20822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.639597 20822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:58.639654 20822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.646246 20822 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:45275
I20260923 15:08:58.646787 20822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:08:58.653959 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 20822
I20260923 15:08:58.654080 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:08:58.654346 8169 external_mini_cluster.cc:1428] Setting key b59c84c208ec1957df72f11a0390a6b6
I20260923 15:08:58.658492 20951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:45275 every 8 connection(s)
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:58.671551 20680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:58.659979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:50105 (local address 127.7.250.126:37969)
0923 15:08:58.660027 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:58.660029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:58.660253 (+ 224us) server_negotiation.cc:407] Connection header received
0923 15:08:58.660995 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:58.660997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:58.661012 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:58.661105 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:58.661945 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.662839 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.664493 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.664805 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.664898 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:58.667932 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:58.667934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:58.667934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:58.667942 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:58.669885 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:58.670441 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:58.670443 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:58.670443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:58.670508 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:58.670795 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:58.670796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:58.670797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:58.670936 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:58.670979 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:58.671375 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 15:08:58.671463 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:08:58.672369 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:58.659665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37969 (local address 127.7.250.67:50105)
0923 15:08:58.660102 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:58.660153 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:58.660846 (+ 693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:58.661202 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:58.661207 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:58.661482 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:58.661822 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:58.661829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.662953 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.662955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:58.664378 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:58.664381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:58.664927 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:58.664928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:58.665075 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:58.665780 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:58.665791 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:58.667830 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:58.669977 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:58.669985 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:58.669987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:58.670340 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:58.670582 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:58.670582 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:58.670583 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:58.670717 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:58.671025 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:58.671031 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:58.671211 (+ 180us) client_negotiation.cc:769] Sending connection context
0923 15:08:58.671915 (+ 704us) client_negotiation.cc:240] Negotiation successful
0923 15:08:58.672102 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":111,"threads_started":1}
I20260923 15:08:58.673106 20952 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37969
I20260923 15:08:58.673272 20952 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:58.673525 20952 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:08:58.673956 20464 ts_manager.cc:194] Registered new tserver with Master: cfcb36195a574b4782990f9d4031f3d8 (127.7.250.67:45275)
I20260923 15:08:58.674506 20464 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:50105
I20260923 15:08:58.677309 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:08:58.678006 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 20545
I20260923 15:08:58.686671 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 20683
I20260923 15:08:58.694468 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 20822
I20260923 15:08:58.699843 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 20431
I20260923 15:08:58.706539 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37969
--webserver_interface=127.7.250.126
--webserver_port=39701
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:37969
--ranger_config_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:08:58.790210 20955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:58.790503 20955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:58.790526 20955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:58.791718 20955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:08:58.791765 20955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:58.791779 20955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:58.791791 20955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:08:58.791805 20955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:08:58.793493 20955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:37969
--ranger_config_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=39701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:58.793748 20955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:58.793982 20955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:58.799187 20963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:58.799259 20960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:58.799199 20961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:58.799433 20955 server_base.cc:1034] running on GCE node
I20260923 15:08:58.799727 20955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:58.800060 20955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:58.801246 20955 hybrid_clock.cc:648] HybridClock initialized: now 1790176138801238 us; error 53 us; skew 500 ppm
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:58.803860 20955 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:08:58.804447 20955 webserver.cc:460] Webserver started at http://127.7.250.126:39701/ using document root <none> and password file <none>
I20260923 15:08:58.804715 20955 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:58.804800 20955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:08:58 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176138, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:58.879174 20955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:08:58.879925 20971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:58.880151 20955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:08:58.880276 20955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "25f82cee2ac147c7b62e832b2bcdb5d9"
format_stamp: "Formatted at 2026-09-23 15:08:57 on dist-test-slave-bb3x"
server_key: "69fa3f0c1b1de6c38fce72f9bd15f655"
server_key_iv: "72136963e0b49da433664a43e8ee7b4d"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:58.880771 20955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:58.905836 20955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:58.908217 20955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:58.908382 20955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:58.913704 20955 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:37969
I20260923 15:08:58.913823 21023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:37969 every 8 connection(s)
I20260923 15:08:58.914279 20955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:08:58.915722 21024 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:08:58.916424 21024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Bootstrap starting.
I20260923 15:08:58.923143 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 20955
I20260923 15:08:58.923517 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:43357
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:37969
--webserver_port=34941
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:08:58.925611 21024 log.cc:824] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:08:58.927558 21024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:08:58.927945 21024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Bootstrap complete.
I20260923 15:08:58.931018 21024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:58.931315 21024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f82cee2ac147c7b62e832b2bcdb5d9, State: Initialized, Role: FOLLOWER
I20260923 15:08:58.931555 21024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:58.931663 21024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:08:58.931736 21024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:08:58.931818 21024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:08:58.932957 21024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:58.933099 21024 leader_election.cc:302] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25f82cee2ac147c7b62e832b2bcdb5d9; no voters:
I20260923 15:08:58.933323 21024 leader_election.cc:288] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:08:58.933363 21028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:08:58.933614 21028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [term 2 LEADER]: Becoming Leader. State: Replica: 25f82cee2ac147c7b62e832b2bcdb5d9, State: Running, Role: LEADER
I20260923 15:08:58.933706 21024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:08:58.933776 21028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } }
I20260923 15:08:58.934111 21028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } } }
I20260923 15:08:58.934285 21028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:08:58.934497 21028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25f82cee2ac147c7b62e832b2bcdb5d9. Latest consensus state: current_term: 2 leader_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f82cee2ac147c7b62e832b2bcdb5d9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37969 } } }
I20260923 15:08:58.934640 21028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:08:58.934804 21036 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:08:58.935459 21036 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:08:58.935683 21036 catalog_manager.cc:1259] Loaded cluster ID: f09286c42b7c4941b5767518a73c81ff
I20260923 15:08:58.935770 21036 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:08:58.935879 21024 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 15:08:58.937434 21036 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:08:58.937693 21036 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Loaded TSK: 0
I20260923 15:08:58.938060 21036 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:08:59.047931 21026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:59.048321 21026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:59.048414 21026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:59.050457 21026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:59.050688 21026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:59.050778 21026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:08:59.053129 21026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:43357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=34941
--tserver_master_addrs=127.7.250.126:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:59.053650 21026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:59.053987 21026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:59.060748 21062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:59.061187 21061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:59.061414 21064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:59.062543 21026 server_base.cc:1034] running on GCE node
I20260923 15:08:59.062772 21026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:59.063038 21026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:59.064208 21026 hybrid_clock.cc:648] HybridClock initialized: now 1790176139064171 us; error 50 us; skew 500 ppm
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:59.067800 21026 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:08:59.068372 21026 webserver.cc:460] Webserver started at http://127.7.250.65:34941/ using document root <none> and password file <none>
I20260923 15:08:59.068660 21026 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:59.068737 21026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:59.099438 21026 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260923 15:08:59.100546 21073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.100782 21026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:08:59.100871 21026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "e93cf8c10c6d892395979e270986d458"
server_key_iv: "dac80c0f82c122c0a664f3c227979522"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:59.101331 21026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:59.112272 21026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:59.119406 21026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:59.119658 21026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:59.119976 21026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:59.120462 21026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:59.122485 21026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:08:59.122617 21026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:59.122691 21026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.129895 21026 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43357
I20260923 15:08:59.130571 21026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:08:59.136687 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21026
I20260923 15:08:59.137058 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:34841
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:37969
--webserver_port=40143
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:08:59.133188 21186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43357 every 8 connection(s)
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:59.150315 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.133888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37969 (local address 127.7.250.65:46187)
0923 15:08:59.136563 (+ 2675us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:59.136619 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:59.137341 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:59.138351 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:59.138355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:59.138583 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:59.138864 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:59.138869 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.140602 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.140603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:59.141860 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:59.141863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.142944 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.142946 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:59.143084 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.143828 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:59.143838 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:59.145365 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:59.148201 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:59.148210 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:59.148213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:59.148540 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:59.148908 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:59.148909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:59.148909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:59.149066 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:59.149581 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:59.149588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:59.149766 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 15:08:59.149863 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 15:08:59.150029 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2617,"thread_start_us":88,"threads_started":1}
I20260923 15:08:59.151528 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.133973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:46187 (local address 127.7.250.126:37969)
0923 15:08:59.137713 (+ 3740us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:59.137733 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:59.137791 (+ 58us) server_negotiation.cc:407] Connection header received
0923 15:08:59.137947 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:59.137954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:59.138011 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:59.138136 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:59.139001 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.140446 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.141982 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.142643 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.142814 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.145517 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:59.145522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:59.145522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:59.145536 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:59.148084 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.148653 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.148656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.148657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.148802 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.149168 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.149169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.149169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.149488 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:59.151068 (+ 1580us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:59.151150 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 15:08:59.151286 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3678,"thread_start_us":109,"threads_started":1}
I20260923 15:08:59.152596 21187 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37969
I20260923 15:08:59.152697 21187 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.152904 21187 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
W20260923 15:08:59.153841 21187 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:37969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 15:08:59.153921 21187 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.154009 21187 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:08:59.154637 21187 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.154726 21187 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
W20260923 15:08:59.155067 21187 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:08:59.275748 21191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:59.276126 21191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:59.276226 21191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:59.278097 21191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:59.278237 21191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:59.278365 21191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:08:59.280694 21191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:34841
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=40143
--tserver_master_addrs=127.7.250.126:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:59.281178 21191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:59.281531 21191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:59.294036 21197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:59.294499 21199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:59.298424 21196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:59.299959 21191 server_base.cc:1034] running on GCE node
I20260923 15:08:59.300238 21191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:59.300541 21191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:59.307874 21191 hybrid_clock.cc:648] HybridClock initialized: now 1790176139307847 us; error 42 us; skew 500 ppm
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:59.310918 21191 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:08:59.311439 21191 webserver.cc:460] Webserver started at http://127.7.250.66:40143/ using document root <none> and password file <none>
I20260923 15:08:59.311699 21191 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:59.311784 21191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:59.327603 21191 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 15:08:59.332013 21207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.332254 21191 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260923 15:08:59.332343 21191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "d5ca166b6d3640058a5062f6557638c4"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "66abe58ae260afdc0d0337dba7a66170"
server_key_iv: "3d700224196568898309ad93423ee402"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:59.332808 21191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:59.345705 21191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:59.348263 21191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:59.348481 21191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:59.348785 21191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:59.349191 21191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:59.349346 21191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.349421 21191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:59.349479 21191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.355641 21191 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:34841
I20260923 15:08:59.355847 21321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:34841 every 8 connection(s)
I20260923 15:08:59.356168 21191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:08:59.360113 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21191
I20260923 15:08:59.360430 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.250.84:48875/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:45275
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:37969
--webserver_port=43815
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:59.370609 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.356957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:53877 (local address 127.7.250.126:37969)
0923 15:08:59.357005 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:59.357007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:59.357472 (+ 465us) server_negotiation.cc:407] Connection header received
0923 15:08:59.358315 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:59.358317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:59.358333 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:59.358372 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:59.359339 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.359918 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.363357 (+ 3439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.363708 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.363775 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.366642 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:59.366644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:59.366644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:59.366655 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:59.368606 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.369176 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.369177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.369178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.369256 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.369581 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.369582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.369583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.369735 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:59.369785 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:59.370364 (+ 579us) server_negotiation.cc:299] Negotiation successful
0923 15:08:59.370467 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:08:59.371510 21324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.357131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37969 (local address 127.7.250.66:53877)
0923 15:08:59.357328 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:59.357378 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:59.358100 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:59.358590 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:59.358595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:59.358867 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:59.359212 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:59.359220 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.361852 (+ 2632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.361854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:59.363251 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:59.363254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.363892 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.363893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:59.364046 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.364672 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:59.364683 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:59.366507 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:59.368711 (+ 2204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:59.368720 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:59.368723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:59.369073 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:59.369341 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:59.369342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:59.369342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:59.369497 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:59.369839 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:59.369846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:59.370049 (+ 203us) client_negotiation.cc:769] Sending connection context
0923 15:08:59.370998 (+ 949us) client_negotiation.cc:240] Negotiation successful
0923 15:08:59.371195 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
I20260923 15:08:59.372305 21322 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37969
I20260923 15:08:59.372463 21322 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.372740 21322 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
W20260923 15:08:59.373442 21322 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:37969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 15:08:59.373555 21322 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.373647 21322 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:08:59.373943 21322 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.374042 21322 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
W20260923 15:08:59.374370 21322 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:08:59.459024 21325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:08:59.459339 21325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:08:59.459386 21325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:08:59.461236 21325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:08:59.461299 21325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:08:59.461342 21325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:08:59.463692 21325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.250.84:48875/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:45275
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=43815
--tserver_master_addrs=127.7.250.126:37969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:08:59.463992 21325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:08:59.464293 21325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:08:59.470252 21333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:59.470327 21330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:08:59.470525 21331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:08:59.470476 21325 server_base.cc:1034] running on GCE node
I20260923 15:08:59.470819 21325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:08:59.471102 21325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:08:59.472280 21325 hybrid_clock.cc:648] HybridClock initialized: now 1790176139472261 us; error 31 us; skew 500 ppm
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:08:59.479405 21325 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:08:59.480037 21325 webserver.cc:460] Webserver started at http://127.7.250.67:43815/ using document root <none> and password file <none>
I20260923 15:08:59.480348 21325 fs_manager.cc:359] Metadata directory not provided
I20260923 15:08:59.480441 21325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:08:59.498749 21325 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260923 15:08:59.499773 21341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.499941 21325 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:08:59.500041 21325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "cfcb36195a574b4782990f9d4031f3d8"
format_stamp: "Formatted at 2026-09-23 15:08:58 on dist-test-slave-bb3x"
server_key: "9fb6aee822c6337df558db3029ba8c9c"
server_key_iv: "7d1286a52e1215d963a0e57f2b45f82b"
server_key_version: "kuduclusterkey@0"
I20260923 15:08:59.500384 21325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:08:59.521804 21325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:08:59.523981 21325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:08:59.524178 21325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:08:59.524453 21325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:08:59.524837 21325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:08:59.524889 21325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.524927 21325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:08:59.524953 21325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:08:59.531610 21325 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:45275
I20260923 15:08:59.532104 21325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:08:59.536736 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21325
I20260923 15:08:59.541654 21455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:45275 every 8 connection(s)
I20260923 15:08:59.548790 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.537445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45430 (local address 127.7.250.126:37969)
0923 15:08:59.537521 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:59.537524 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:59.540408 (+ 2884us) server_negotiation.cc:407] Connection header received
0923 15:08:59.540488 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:59.540489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:59.540506 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:59.540596 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:59.541413 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.542214 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.543872 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.544166 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.544246 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.545747 (+ 1501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:59.545749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:59.545749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:59.545757 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:59.547531 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.547953 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.547954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.547955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.548027 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.548251 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.548252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.548252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.548380 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:59.548562 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:59.548604 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 15:08:59.548687 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Sep 23 15:08:59 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176139, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:08:59.558002 21459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.542323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37969 (local address 127.7.250.67:48485)
0923 15:08:59.544139 (+ 1816us) negotiation.cc:107] Waiting for socket to connect
0923 15:08:59.544192 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 15:08:59.546242 (+ 2050us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:08:59.546588 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:08:59.546592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:08:59.546853 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:08:59.547090 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:59.547096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.547881 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.547882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:59.549077 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:08:59.549080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.549606 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.549607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:08:59.549771 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.550356 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:08:59.550365 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:08:59.551815 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:08:59.556159 (+ 4344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:59.556169 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:59.556172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:59.556503 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:59.556807 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:08:59.556808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:08:59.556808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:08:59.556962 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:08:59.557288 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:08:59.557295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:08:59.557466 (+ 171us) client_negotiation.cc:769] Sending connection context
0923 15:08:59.557577 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:08:59.557755 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1770,"thread_start_us":97,"threads_started":1}
I20260923 15:08:59.558499 21460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:08:59.542643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:48485 (local address 127.7.250.126:37969)
0923 15:08:59.544078 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0923 15:08:59.544080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:08:59.545678 (+ 1598us) server_negotiation.cc:407] Connection header received
0923 15:08:59.546364 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:08:59.546365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:08:59.546379 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:08:59.546448 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:08:59.547190 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.547771 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.549175 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:08:59.549498 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:08:59.549580 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:08:59.551932 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:08:59.551933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:08:59.551933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:08:59.551942 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:08:59.556045 (+ 4103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.556598 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.556599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.556599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.556672 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:08:59.557050 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:08:59.557052 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:08:59.557052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:08:59.557199 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:08:59.557262 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:08:59.558290 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0923 15:08:59.558392 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405,"thread_start_us":63,"threads_started":1}
I20260923 15:08:59.558840 21456 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37969
I20260923 15:08:59.558947 21456 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.559155 21456 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
W20260923 15:08:59.559818 21456 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:37969 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 15:08:59.559890 21456 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.559957 21456 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:08:59.560261 21456 heartbeater.cc:464] Registering TS with master...
I20260923 15:08:59.560339 21456 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
W20260923 15:08:59.560840 21456 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:09:00.155241 21187 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:00.155423 21187 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:09:00.155839 20988 ts_manager.cc:194] Registered new tserver with Master: 0b6ee3a4ffbb41469c97600e9b5c42ad (127.7.250.65:43357)
Sep 23 15:09:00 dist-test-slave-bb3x krb5kdc[19983](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790176140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:00.156817 20988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:46187
I20260923 15:09:00.374583 21322 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:00.374752 21322 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:09:00.375087 20988 ts_manager.cc:194] Registered new tserver with Master: d5ca166b6d3640058a5062f6557638c4 (127.7.250.66:34841)
I20260923 15:09:00.375669 20988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:53877
Sep 23 15:09:00 dist-test-slave-bb3x krb5kdc[19983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176140, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for HTTP/127.7.250.84@KRBTEST.COM
I20260923 15:09:00.561028 21456 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:00.561168 21456 heartbeater.cc:511] Master 127.7.250.126:37969 requested a full tablet report, sending...
I20260923 15:09:00.561453 20988 ts_manager.cc:194] Registered new tserver with Master: cfcb36195a574b4782990f9d4031f3d8 (127.7.250.67:45275)
I20260923 15:09:00.561955 20988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:48485
I20260923 15:09:00.572933 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:09:00.582603 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.573936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45442 (local address 127.7.250.126:37969)
0923 15:09:00.574230 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.574232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.574419 (+ 187us) server_negotiation.cc:407] Connection header received
0923 15:09:00.574558 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.574560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.574581 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.574706 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:00.575564 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.576274 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.577280 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.577568 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.577638 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.579213 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:00.579214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:00.579215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:00.579222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:00.581145 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:00.581535 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:00.581536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:00.581537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:00.581602 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:00.581826 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:00.581827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:00.581827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:00.581968 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:00.582064 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.582319 (+ 255us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.582408 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":66,"threads_started":1}
I20260923 15:09:00.585438 20988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:09:00.587699 20988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:09:00.613936 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.608984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:34841 (local address 127.0.0.1:47802)
0923 15:09:00.609761 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.609802 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.609864 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.610151 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.610153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.610156 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.610226 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.610228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.611487 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.611488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.612695 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.612698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.613645 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.613646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.613720 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.613733 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.613798 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.613848 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":82,"threads_started":1}
I20260923 15:09:00.614238 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.609232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47802 (local address 127.7.250.66:34841)
0923 15:09:00.609352 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.609358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.609942 (+ 584us) server_negotiation.cc:407] Connection header received
0923 15:09:00.609979 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.609981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.609997 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.610060 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.610353 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.611396 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.612788 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.613479 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.613555 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.613611 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.614123 (+ 512us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.614165 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":44,"threads_started":1}
I20260923 15:09:00.614964 21256 tablet_service.cc:1467] Processing CreateTablet for tablet d4356469fd444618b50616faa7189013 (DEFAULT_TABLE table=test-table [id=d66d586aa9f8402185661eb1525a50bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:00.615341 21256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4356469fd444618b50616faa7189013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:00.615710 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.608986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43357 (local address 127.0.0.1:56438)
0923 15:09:00.610105 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.610124 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.610172 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.610614 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.610614 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.610617 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.610664 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.610665 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.612215 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.612215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.613554 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.613557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.615471 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.615472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.615533 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.615537 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.615593 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.615634 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1102,"thread_start_us":106,"threads_started":1}
I20260923 15:09:00.620924 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.609239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56438 (local address 127.7.250.65:43357)
0923 15:09:00.610310 (+ 1071us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.610315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.610324 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:09:00.610359 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.610361 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.610377 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.610452 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.610746 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.612130 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.613656 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.615284 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.615353 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.620764 (+ 5411us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.620793 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.620838 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1009,"thread_start_us":63,"threads_started":1}
I20260923 15:09:00.621680 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.609463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:45275 (local address 127.0.0.1:37772)
0923 15:09:00.616011 (+ 6548us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.616021 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.616057 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.616367 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.616368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.616370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.616400 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.616401 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.618332 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.618333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.619381 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.619385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.621435 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.621436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.621499 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.621503 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.621559 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.621602 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6536,"thread_start_us":80,"threads_started":1}
I20260923 15:09:00.621860 21120 tablet_service.cc:1467] Processing CreateTablet for tablet d4356469fd444618b50616faa7189013 (DEFAULT_TABLE table=test-table [id=d66d586aa9f8402185661eb1525a50bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:00.622093 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.609668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37772 (local address 127.7.250.67:45275)
0923 15:09:00.610577 (+ 909us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.610582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.616137 (+ 5555us) server_negotiation.cc:407] Connection header received
0923 15:09:00.616181 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.616183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.616200 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.616273 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.616477 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.618205 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.619485 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.620301 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.620384 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.621176 (+ 792us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.621901 (+ 725us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.621948 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":876,"thread_start_us":56,"threads_started":1}
I20260923 15:09:00.622226 21120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4356469fd444618b50616faa7189013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:00.622962 21390 tablet_service.cc:1467] Processing CreateTablet for tablet d4356469fd444618b50616faa7189013 (DEFAULT_TABLE table=test-table [id=d66d586aa9f8402185661eb1525a50bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:00.623291 21390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4356469fd444618b50616faa7189013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:00.625658 21490 tablet_bootstrap.cc:492] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: Bootstrap starting.
I20260923 15:09:00.626117 21491 tablet_bootstrap.cc:492] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Bootstrap starting.
I20260923 15:09:00.626626 21490 tablet_bootstrap.cc:654] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:00.626747 21491 tablet_bootstrap.cc:654] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:00.627055 21491 log.cc:824] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:00.627095 21490 log.cc:824] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:00.627799 21491 tablet_bootstrap.cc:492] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: No bootstrap required, opened a new log
I20260923 15:09:00.627872 21491 ts_tablet_manager.cc:1397] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:09:00.630290 21489 tablet_bootstrap.cc:492] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: Bootstrap starting.
I20260923 15:09:00.630524 21491 raft_consensus.cc:348] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.630709 21491 raft_consensus.cc:374] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:00.630755 21491 raft_consensus.cc:729] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfcb36195a574b4782990f9d4031f3d8, State: Initialized, Role: FOLLOWER
I20260923 15:09:00.630915 21491 consensus_queue.cc:260] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.631178 21491 ts_tablet_manager.cc:1428] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:09:00.631186 21489 tablet_bootstrap.cc:654] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:00.631300 21456 heartbeater.cc:503] Master 127.7.250.126:37969 was elected leader, sending a full tablet report...
I20260923 15:09:00.631565 21489 log.cc:824] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:00.632534 21489 tablet_bootstrap.cc:492] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: No bootstrap required, opened a new log
I20260923 15:09:00.632553 21490 tablet_bootstrap.cc:492] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: No bootstrap required, opened a new log
I20260923 15:09:00.632596 21489 ts_tablet_manager.cc:1397] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:09:00.632620 21490 ts_tablet_manager.cc:1397] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260923 15:09:00.635023 21489 raft_consensus.cc:348] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.635245 21489 raft_consensus.cc:374] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:00.635316 21489 raft_consensus.cc:729] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ca166b6d3640058a5062f6557638c4, State: Initialized, Role: FOLLOWER
I20260923 15:09:00.635504 21489 consensus_queue.cc:260] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.635799 21489 ts_tablet_manager.cc:1428] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:09:00.635936 21322 heartbeater.cc:503] Master 127.7.250.126:37969 was elected leader, sending a full tablet report...
I20260923 15:09:00.635813 21490 raft_consensus.cc:348] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.636018 21490 raft_consensus.cc:374] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:00.636075 21490 raft_consensus.cc:729] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6ee3a4ffbb41469c97600e9b5c42ad, State: Initialized, Role: FOLLOWER
I20260923 15:09:00.636211 21490 consensus_queue.cc:260] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.636822 21490 ts_tablet_manager.cc:1428] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260923 15:09:00.636936 21187 heartbeater.cc:503] Master 127.7.250.126:37969 was elected leader, sending a full tablet report...
I20260923 15:09:00.637627 21496 raft_consensus.cc:482] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:00.637764 21496 raft_consensus.cc:504] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.638149 21496 leader_election.cc:288] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b6ee3a4ffbb41469c97600e9b5c42ad (127.7.250.65:43357), d5ca166b6d3640058a5062f6557638c4 (127.7.250.66:34841)
I20260923 15:09:00.642439 21498 raft_consensus.cc:482] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:00.642534 21498 raft_consensus.cc:504] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.642591 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.638690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:54505 (local address 127.7.250.66:34841)
0923 15:09:00.638730 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.638732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.639051 (+ 319us) server_negotiation.cc:407] Connection header received
0923 15:09:00.639110 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.639111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.639121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.639174 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.639381 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.640079 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.641586 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.642156 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.642205 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.642214 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.642467 (+ 253us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.642510 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:09:00.643013 21499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.638621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:34841 (local address 127.7.250.67:54505)
0923 15:09:00.638971 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.638983 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.639045 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.639226 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.639227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.639230 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.639307 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.639309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.640181 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.640181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.641450 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.641453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.642317 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.642318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.642387 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.642391 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.642864 (+ 473us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.642911 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":81,"threads_started":1}
I20260923 15:09:00.643447 21276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4356469fd444618b50616faa7189013" candidate_uuid: "cfcb36195a574b4782990f9d4031f3d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ca166b6d3640058a5062f6557638c4" is_pre_election: true
I20260923 15:09:00.643611 21276 raft_consensus.cc:2454] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfcb36195a574b4782990f9d4031f3d8 in term 0.
I20260923 15:09:00.643649 21498 leader_election.cc:288] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5ca166b6d3640058a5062f6557638c4 (127.7.250.66:34841), cfcb36195a574b4782990f9d4031f3d8 (127.7.250.67:45275)
I20260923 15:09:00.643945 21344 leader_election.cc:302] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cfcb36195a574b4782990f9d4031f3d8, d5ca166b6d3640058a5062f6557638c4; no voters:
I20260923 15:09:00.644093 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.638398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:53819 (local address 127.7.250.65:43357)
0923 15:09:00.638441 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.638444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.639182 (+ 738us) server_negotiation.cc:407] Connection header received
0923 15:09:00.639268 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.639269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.639279 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.639307 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.639509 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.640223 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.641679 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.642242 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.642331 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.642339 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.643966 (+ 1627us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.644017 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:09:00.644333 21494 raft_consensus.cc:2781] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:09:00.644538 21494 raft_consensus.cc:482] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:09:00.644605 21494 raft_consensus.cc:3037] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:00.645483 21494 raft_consensus.cc:504] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.644498 21500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.638847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43357 (local address 127.7.250.67:53819)
0923 15:09:00.639136 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.639143 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.639202 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.639367 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.639367 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.639369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.639441 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.639442 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.640324 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.640325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.641531 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.641534 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.643796 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.643796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.643877 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.643881 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.644199 (+ 318us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.644281 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":72,"threads_started":1}
I20260923 15:09:00.646008 21141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4356469fd444618b50616faa7189013" candidate_uuid: "cfcb36195a574b4782990f9d4031f3d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" is_pre_election: true
I20260923 15:09:00.646104 21141 raft_consensus.cc:2454] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfcb36195a574b4782990f9d4031f3d8 in term 0.
I20260923 15:09:00.646569 21141 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4356469fd444618b50616faa7189013" candidate_uuid: "cfcb36195a574b4782990f9d4031f3d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad"
I20260923 15:09:00.646670 21141 raft_consensus.cc:3037] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:00.647459 21141 raft_consensus.cc:2454] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfcb36195a574b4782990f9d4031f3d8 in term 1.
I20260923 15:09:00.647687 21342 leader_election.cc:302] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6ee3a4ffbb41469c97600e9b5c42ad, cfcb36195a574b4782990f9d4031f3d8; no voters:
I20260923 15:09:00.647799 21496 raft_consensus.cc:2781] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:00.647950 21496 raft_consensus.cc:686] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 1 LEADER]: Becoming Leader. State: Replica: cfcb36195a574b4782990f9d4031f3d8, State: Running, Role: LEADER
I20260923 15:09:00.648067 21496 consensus_queue.cc:237] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } }
I20260923 15:09:00.648912 20987 catalog_manager.cc:5495] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfcb36195a574b4782990f9d4031f3d8 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "cfcb36195a574b4782990f9d4031f3d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfcb36195a574b4782990f9d4031f3d8" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 45275 } health_report { overall_health: HEALTHY } } }
I20260923 15:09:00.649780 21276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4356469fd444618b50616faa7189013" candidate_uuid: "cfcb36195a574b4782990f9d4031f3d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ca166b6d3640058a5062f6557638c4"
I20260923 15:09:00.649870 21276 raft_consensus.cc:3037] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:00.650753 21276 raft_consensus.cc:2454] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfcb36195a574b4782990f9d4031f3d8 in term 1.
I20260923 15:09:00.650995 21494 leader_election.cc:288] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [CANDIDATE]: Term 1 election: Requested vote from peers 0b6ee3a4ffbb41469c97600e9b5c42ad (127.7.250.65:43357), d5ca166b6d3640058a5062f6557638c4 (127.7.250.66:34841)
I20260923 15:09:00.651160 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.643206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:46439 (local address 127.7.250.66:34841)
0923 15:09:00.643257 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.643259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.644465 (+ 1206us) server_negotiation.cc:407] Connection header received
0923 15:09:00.644958 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.644959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.644969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.644996 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.647357 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.648031 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.649312 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.649822 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.649877 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.649888 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.651031 (+ 1143us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.651091 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:09:00.651412 21501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.643003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:34841 (local address 127.7.250.65:46439)
0923 15:09:00.644402 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.644413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.644637 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.645046 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.645046 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.645047 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.645121 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.645122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.648151 (+ 3029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.648152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.649200 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.649203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.649969 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.649970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.650034 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.650038 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.651270 (+ 1232us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.651318 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1381,"thread_start_us":88,"threads_started":1}
I20260923 15:09:00.651618 21276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4356469fd444618b50616faa7189013" candidate_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ca166b6d3640058a5062f6557638c4" is_pre_election: true
I20260923 15:09:00.651718 21276 raft_consensus.cc:2379] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0b6ee3a4ffbb41469c97600e9b5c42ad in current term 1: Already voted for candidate cfcb36195a574b4782990f9d4031f3d8 in this term.
I20260923 15:09:00.652190 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.643595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:49919 (local address 127.7.250.67:45275)
0923 15:09:00.643630 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.643632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.644562 (+ 930us) server_negotiation.cc:407] Connection header received
0923 15:09:00.644711 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.644712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.644722 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.644755 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:00.645195 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.645866 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.650140 (+ 4274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.650742 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.650791 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.650800 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.652080 (+ 1280us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.652130 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:09:00.652429 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.643435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:45275 (local address 127.7.250.65:49919)
0923 15:09:00.644511 (+ 1076us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.644520 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:00.644590 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:00.644814 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:00.644815 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:00.644817 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:00.644881 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.644882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.646005 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.646006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.647244 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:00.647248 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.650862 (+ 3614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.650863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:00.650941 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.650944 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:09:00.652306 (+ 1362us) client_negotiation.cc:240] Negotiation successful
0923 15:09:00.652353 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1061,"thread_start_us":80,"threads_started":1}
I20260923 15:09:00.652688 21410 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4356469fd444618b50616faa7189013" candidate_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfcb36195a574b4782990f9d4031f3d8" is_pre_election: true
I20260923 15:09:00.653002 21075 leader_election.cc:302] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0b6ee3a4ffbb41469c97600e9b5c42ad; no voters: cfcb36195a574b4782990f9d4031f3d8, d5ca166b6d3640058a5062f6557638c4
I20260923 15:09:00.653172 21498 raft_consensus.cc:2726] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 15:09:00.659788 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.656661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.7.250.67:45275)
0923 15:09:00.656695 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:00.656696 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:00.656703 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:09:00.656732 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:00.656733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:00.656752 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:00.656813 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:00.657000 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.657715 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.658848 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:00.659125 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:00.659194 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:00.659402 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:00.659533 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:09:00.659643 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:00.659687 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 15:09:00.659707 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:09:00.663199 21276 raft_consensus.cc:1261] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 1 FOLLOWER]: Refusing update from remote peer cfcb36195a574b4782990f9d4031f3d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:00.663241 21141 raft_consensus.cc:1261] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 1 FOLLOWER]: Refusing update from remote peer cfcb36195a574b4782990f9d4031f3d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:00.663533 21503 consensus_queue.cc:1035] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5ca166b6d3640058a5062f6557638c4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 34841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:00.663681 21494 consensus_queue.cc:1035] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6ee3a4ffbb41469c97600e9b5c42ad" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:00.666970 21508 mvcc.cc:205] Tried to move back new op lower bound from 7332561472153739264 to 7332561472095297536. Current Snapshot: MvccSnapshot[applied={T|T < 7332561472153739264}]
I20260923 15:09:00.668238 21507 mvcc.cc:205] Tried to move back new op lower bound from 7332561472153739264 to 7332561472095297536. Current Snapshot: MvccSnapshot[applied={T|T < 7332561472153739264}]
I20260923 15:09:00.675439 20988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:00.675587 20988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:00.676481 20988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f82cee2ac147c7b62e832b2bcdb5d9: Sending DeleteTablet for 3 replicas of tablet d4356469fd444618b50616faa7189013
I20260923 15:09:00.676913 21256 tablet_service.cc:1514] Processing DeleteTablet for tablet d4356469fd444618b50616faa7189013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:00 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:47802
I20260923 15:09:00.676980 21390 tablet_service.cc:1514] Processing DeleteTablet for tablet d4356469fd444618b50616faa7189013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:00 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:37772
I20260923 15:09:00.677206 21120 tablet_service.cc:1514] Processing DeleteTablet for tablet d4356469fd444618b50616faa7189013 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:00 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:56438
I20260923 15:09:00.677459 21517 tablet_replica.cc:330] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad: stopping tablet replica
I20260923 15:09:00.677575 21517 raft_consensus.cc:2229] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:00.677732 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21026
I20260923 15:09:00.677755 21517 raft_consensus.cc:2258] T d4356469fd444618b50616faa7189013 P 0b6ee3a4ffbb41469c97600e9b5c42ad [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 15:09:00.683591 20972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:43357 (error 108)
I20260923 15:09:00.683811 21516 tablet_replica.cc:330] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4: stopping tablet replica
I20260923 15:09:00.683954 21516 raft_consensus.cc:2229] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:00.684015 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21191
I20260923 15:09:00.684111 21516 raft_consensus.cc:2258] T d4356469fd444618b50616faa7189013 P d5ca166b6d3640058a5062f6557638c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:00.684288 21515 tablet_replica.cc:330] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: stopping tablet replica
I20260923 15:09:00.684383 21515 raft_consensus.cc:2229] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:09:00.684664 21515 raft_consensus.cc:2258] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:00.685299 21515 ts_tablet_manager.cc:1905] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:00.687250 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.685329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43357 (local address 127.0.0.1:56452)
0923 15:09:00.687152 (+ 1823us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.687212 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1798}
W20260923 15:09:00.687381 20972 catalog_manager.cc:4584] TS 0b6ee3a4ffbb41469c97600e9b5c42ad (127.7.250.65:43357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4356469fd444618b50616faa7189013: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43357: connect: Connection refused (error 111)
I20260923 15:09:00.687793 21515 ts_tablet_manager.cc:1918] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:00.687872 21515 log.cc:1196] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/d4356469fd444618b50616faa7189013
I20260923 15:09:00.688175 21515 ts_tablet_manager.cc:1939] T d4356469fd444618b50616faa7189013 P cfcb36195a574b4782990f9d4031f3d8: Deleting consensus metadata
I20260923 15:09:00.688589 20973 catalog_manager.cc:4841] TS cfcb36195a574b4782990f9d4031f3d8 (127.7.250.67:45275): tablet d4356469fd444618b50616faa7189013 (table test-table [id=d66d586aa9f8402185661eb1525a50bf]) successfully deleted
I20260923 15:09:00.689829 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21325
I20260923 15:09:00.690441 21484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:00.689912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:34841 (local address 127.0.0.1:47818)
0923 15:09:00.690370 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:00.690400 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.66:34841: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441}
I20260923 15:09:00.696058 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 20955
I20260923 15:09:00.701162 8169 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 15:09:01.046554 8169 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 15:09:01.046653 8169 mini_ranger.cc:66] Stopping Ranger...
I20260923 15:09:01.391685 8169 mini_ranger.cc:68] Stopped Ranger
2026-09-23 15:09:01.391 UTC [20014] LOG: received smart shutdown request
2026-09-23 15:09:01.396 UTC [20014] LOG: background worker "logical replication launcher" (PID 20023) exited with exit code 1
2026-09-23 15:09:01.397 UTC [20018] LOG: shutting down
2026-09-23 15:09:01.610 UTC [20014] LOG: database system is shut down
2026-09-23T15:09:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49197 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:09:01 dist-test-slave-bb3x krb5kdc[21527](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:09:01 dist-test-slave-bb3x krb5kdc[21527](info): set up 2 sockets
Sep 23 15:09:01 dist-test-slave-bb3x krb5kdc[21527](info): commencing operation
krb5kdc: starting...
W20260923 15:09:02.819779 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:09:02 dist-test-slave-bb3x krb5kdc[21527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:09:02.946230 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44171
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:44171,127.7.250.125:43769,127.7.250.124:44623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:03.028265 21540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:03.028522 21540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:03.028560 21540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:03.029733 21540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:03.029810 21540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:03.029834 21540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:03.029856 21540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:03.029884 21540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:03.031442 21540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:44171,127.7.250.125:43769,127.7.250.124:44623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:03.031745 21540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:03.031978 21540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:03.037015 21548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:03.037034 21545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:03.037096 21546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:03.037256 21540 server_base.cc:1034] running on GCE node
I20260923 15:09:03.037530 21540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:03.037858 21540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:03.039108 21540 hybrid_clock.cc:648] HybridClock initialized: now 1790176143039095 us; error 53 us; skew 500 ppm
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:03.045778 21540 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:03.046487 21540 webserver.cc:460] Webserver started at http://127.7.250.126:37739/ using document root <none> and password file <none>
I20260923 15:09:03.046757 21540 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:03.046832 21540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:03.047003 21540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:03.048223 21540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "9d11606b8b694f47b3342ccebd9571ef"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "935fee02cc160e567afe731976b92ebe"
server_key_iv: "125160b890300af7a9e4eaac96d95a7d"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.048616 21540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "9d11606b8b694f47b3342ccebd9571ef"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "935fee02cc160e567afe731976b92ebe"
server_key_iv: "125160b890300af7a9e4eaac96d95a7d"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.050177 21540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:09:03.051117 21555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:03.051390 21540 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:03.051481 21540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "9d11606b8b694f47b3342ccebd9571ef"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "935fee02cc160e567afe731976b92ebe"
server_key_iv: "125160b890300af7a9e4eaac96d95a7d"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.051584 21540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:03.062813 21540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:03.065223 21540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:03.065390 21540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:03.069291 21540 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:44171
I20260923 15:09:03.069345 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:44171 every 8 connection(s)
I20260923 15:09:03.069732 21540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:03.070756 21608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:03.071741 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21540
I20260923 15:09:03.071869 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:09:03.072077 8169 external_mini_cluster.cc:1428] Setting key b975c428e63c247c50d459335c930494
I20260923 15:09:03.072971 21608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.250.125@KRBTEST.COM; defaulting to no policy
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.125@KRBTEST.COM" created.
I20260923 15:09:03.083312 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.073918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44171 (local address 127.0.0.1:36130)
0923 15:09:03.074116 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.074155 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.074706 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.075020 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.075022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.075303 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.075463 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.075467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.076536 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.076537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.077215 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.077217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.077750 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.077755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.077827 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.078443 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.078451 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.079961 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.081763 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.081770 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.081773 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.081962 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.082226 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.082227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.082227 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.082349 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.082726 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.082731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.082795 (+ 64us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.082895 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.083117 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":77,"thread_start_us":114,"threads_started":1}
I20260923 15:09:03.083320 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.073938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36130 (local address 127.7.250.126:44171)
0923 15:09:03.074359 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.074363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.074400 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:09:03.074877 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.074881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.074910 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.075001 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.075564 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.076422 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.077320 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.077644 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.077765 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.080073 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.080081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.080081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.080088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.081659 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.082059 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.082062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.082063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.082147 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.082414 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.082415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.082416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.082644 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.082737 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.082934 (+ 197us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.083056 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":104,"threads_started":1}
I20260923 15:09:03.084352 21608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:09:03.084733 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.084605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42900)
0923 15:09:03.084656 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.084701 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 15:09:03.084863 21557 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
I20260923 15:09:03.085273 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.085178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42910)
0923 15:09:03.085227 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.085251 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 15:09:03.085378 21608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:43769: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab.
Entry for principal kudu/127.7.250.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab.
I20260923 15:09:03.101652 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:43769
--webserver_interface=127.7.250.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab
--principal=kudu/127.7.250.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:44171,127.7.250.125:43769,127.7.250.124:44623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:03.134631 21608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } attempt: 1
I20260923 15:09:03.135246 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.135031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42926)
0923 15:09:03.135175 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.135213 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260923 15:09:03.135689 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.135531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42938)
0923 15:09:03.135596 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.135648 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 15:09:03.135849 21608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:43769: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
W20260923 15:09:03.189869 21615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:03.190110 21615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:03.190137 21615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:03.191478 21615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:03.191541 21615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:03.191560 21615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:03.191574 21615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:03.191588 21615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:03.193269 21615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:44171,127.7.250.125:43769,127.7.250.124:44623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab
--principal=kudu/127.7.250.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:43769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.250.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:03.193558 21615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:03.193821 21615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:03.199060 21621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:03.199093 21620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:03.199231 21615 server_base.cc:1034] running on GCE node
W20260923 15:09:03.199148 21623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:03.199551 21615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:03.199882 21615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:03.201062 21615 hybrid_clock.cc:648] HybridClock initialized: now 1790176143201049 us; error 43 us; skew 500 ppm
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:03.203789 21615 init.cc:377] Logged in from keytab as kudu/127.7.250.125@KRBTEST.COM (short username kudu)
I20260923 15:09:03.204051 21608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } attempt: 2
I20260923 15:09:03.204310 21615 webserver.cc:460] Webserver started at http://127.7.250.125:44395/ using document root <none> and password file <none>
I20260923 15:09:03.204537 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.204381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42950)
0923 15:09:03.204452 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.204493 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 15:09:03.204634 21615 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:03.204716 21615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:03.204864 21615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:03.204892 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.204798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42954)
0923 15:09:03.204833 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.204864 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 15:09:03.204994 21608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:43769: Network error: Client connection negotiation failed: client connection to 127.7.250.125:43769: connect: Connection refused (error 111)
I20260923 15:09:03.205993 21615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data/instance:
uuid: "91e402d8e8a34c13b67c615f79c9f95c"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "0cd42246c2b92dd14a3261fdf4c74504"
server_key_iv: "dd16d42180574b20fc7a0303fcbcb1d1"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.206375 21615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal/instance:
uuid: "91e402d8e8a34c13b67c615f79c9f95c"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "0cd42246c2b92dd14a3261fdf4c74504"
server_key_iv: "dd16d42180574b20fc7a0303fcbcb1d1"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.207826 21615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:03.208616 21630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:03.208832 21615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:03.208940 21615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal
uuid: "91e402d8e8a34c13b67c615f79c9f95c"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "0cd42246c2b92dd14a3261fdf4c74504"
server_key_iv: "dd16d42180574b20fc7a0303fcbcb1d1"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.209017 21615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:03.230161 21615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:03.232667 21615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:03.232868 21615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:03.237557 21615 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.125:43769
I20260923 15:09:03.237613 21683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.125:43769 every 8 connection(s)
I20260923 15:09:03.237994 21615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
I20260923 15:09:03.238971 21684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:03.240583 21684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:03.247339 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21615
I20260923 15:09:03.247506 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-1/wal/instance
I20260923 15:09:03.247774 8169 external_mini_cluster.cc:1428] Setting key 26fe086ce89307fb60184bd7deed6f2e
I20260923 15:09:03.250998 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.241245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44171 (local address 127.0.0.1:36134)
0923 15:09:03.241415 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.241460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.241923 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.242234 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.242238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.242575 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.242828 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.242832 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.243553 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.243554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.244434 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.244436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.244827 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.244828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.244934 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.245499 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.245508 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.247037 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.249250 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.249260 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.249263 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.249611 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.249894 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.249895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.249896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.250049 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.250376 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.250383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.250559 (+ 176us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.250693 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.250825 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":106,"threads_started":1}
I20260923 15:09:03.251083 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.241339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36134 (local address 127.7.250.126:44171)
0923 15:09:03.241403 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.241405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.241556 (+ 151us) server_negotiation.cc:407] Connection header received
0923 15:09:03.242005 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.242006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.242016 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.242144 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.242940 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.243455 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.244520 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.244747 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.244806 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.247192 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.247194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.247194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.247203 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.249137 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.249747 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.249748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.249749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.249816 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.250122 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.250122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.250123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.250252 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.250379 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.250742 (+ 363us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.250852 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:09:03.251812 21684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.124@KRBTEST.COM; defaulting to no policy
I20260923 15:09:03.260015 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.252077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42962)
0923 15:09:03.252126 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.252136 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.252173 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.252446 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.252447 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.252635 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.252711 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.252714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.253552 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.253553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.254201 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.254203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.255014 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.255015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.255079 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.255560 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.255566 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.257064 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.259047 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.259047 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.259048 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.259220 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.259407 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.259408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.259408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.259460 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.259759 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.259760 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.259800 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.259857 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.259939 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 15:09:03.260102 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.252149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.7.250.125:43769)
0923 15:09:03.252293 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.252297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.252314 (+ 17us) server_negotiation.cc:407] Connection header received
0923 15:09:03.252342 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.252344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.252354 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.252408 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.252796 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.253469 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.254316 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.254865 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.254943 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.257163 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.257172 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.257173 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.257180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.258967 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.259294 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.259297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.259297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.259349 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.259529 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.259529 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.259530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.259695 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.259785 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.259880 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.259996 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.124@KRBTEST.COM" created.
I20260923 15:09:03.261003 21684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:09:03.261363 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.261244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:44754)
0923 15:09:03.261293 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.261339 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:09:03.261493 21632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
I20260923 15:09:03.261925 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.261827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:44758)
0923 15:09:03.261876 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.261904 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:09:03.262104 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44623: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab.
Entry for principal kudu/127.7.250.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab.
I20260923 15:09:03.278196 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:44623
--webserver_interface=127.7.250.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab
--principal=kudu/127.7.250.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:41767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:44171,127.7.250.125:43769,127.7.250.124:44623
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:03.296155 21608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } attempt: 3
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:09:03.303939 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.296557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:42970)
0923 15:09:03.296632 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.296651 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.296721 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.296875 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.296875 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.297038 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.297125 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.297127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.297860 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.297862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.298456 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.298458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.298844 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.298845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.298925 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.299408 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.299413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.300835 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.302836 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.302837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.302838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.303073 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.303281 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.303281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.303281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.303336 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.303639 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.303640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.303697 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.303771 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.303852 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260923 15:09:03.303995 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.296656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.7.250.125:43769)
0923 15:09:03.296725 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.296727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.296748 (+ 21us) server_negotiation.cc:407] Connection header received
0923 15:09:03.296777 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.296778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.296788 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.296867 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.297224 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.297745 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.298541 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.298756 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.298844 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.300926 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.300927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.300928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.300937 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.302746 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.303149 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.303150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.303150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.303215 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.303396 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.303397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.303397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.303546 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.303615 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.303813 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.303903 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:09:03.304440 21608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:09:03.304862 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.304750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:44770)
0923 15:09:03.304805 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.304839 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:09:03.305263 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.305175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:44778)
0923 15:09:03.305218 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.305244 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:09:03.305356 21608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44623: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
I20260923 15:09:03.311348 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } attempt: 1
I20260923 15:09:03.311817 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.311665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:44786)
0923 15:09:03.311736 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.311778 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 15:09:03.312242 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.312128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:44798)
0923 15:09:03.312185 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.312221 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 15:09:03.312357 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44623: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
I20260923 15:09:03.336527 21608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } attempt: 1
I20260923 15:09:03.337172 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.337003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49466)
0923 15:09:03.337079 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.337133 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 15:09:03.337580 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.337472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49478)
0923 15:09:03.337521 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.337554 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 15:09:03.337718 21608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44623: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
W20260923 15:09:03.366297 21691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:03.366518 21691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:03.366544 21691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:03.367654 21691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:03.367697 21691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:03.367713 21691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:03.367728 21691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:03.367743 21691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:03.369202 21691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:41767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:44171,127.7.250.125:43769,127.7.250.124:44623
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab
--principal=kudu/127.7.250.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:44623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.250.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:03.369401 21691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:03.369583 21691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:03.374843 21697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:03.374871 21696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:03.374891 21691 server_base.cc:1034] running on GCE node
W20260923 15:09:03.374871 21699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:03.375373 21691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:03.375723 21691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:03.376899 21691 hybrid_clock.cc:648] HybridClock initialized: now 1790176143376865 us; error 53 us; skew 500 ppm
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:03.379467 21691 init.cc:377] Logged in from keytab as kudu/127.7.250.124@KRBTEST.COM (short username kudu)
I20260923 15:09:03.379938 21691 webserver.cc:460] Webserver started at http://127.7.250.124:34025/ using document root <none> and password file <none>
I20260923 15:09:03.380172 21691 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:03.380220 21691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:03.380345 21691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:03.380476 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } attempt: 2
I20260923 15:09:03.380901 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.380776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49482)
0923 15:09:03.380837 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.380873 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 15:09:03.381304 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.381214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49494)
0923 15:09:03.381258 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.381282 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:09:03.381438 21684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44623: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44623: connect: Connection refused (error 111)
I20260923 15:09:03.381563 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data/instance:
uuid: "367a97d518604fc79dd01d1d14ae6d85"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "620b1a783aff63dcbd87dece35459f73"
server_key_iv: "483d23100612d63737e8ebb7c90b225c"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.381879 21691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal/instance:
uuid: "367a97d518604fc79dd01d1d14ae6d85"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "620b1a783aff63dcbd87dece35459f73"
server_key_iv: "483d23100612d63737e8ebb7c90b225c"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.383297 21691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:09:03.384228 21706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:03.384485 21691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:03.384591 21691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal
uuid: "367a97d518604fc79dd01d1d14ae6d85"
format_stamp: "Formatted at 2026-09-23 15:09:03 on dist-test-slave-bb3x"
server_key: "620b1a783aff63dcbd87dece35459f73"
server_key_iv: "483d23100612d63737e8ebb7c90b225c"
server_key_version: "encryptionkey@0"
I20260923 15:09:03.384682 21691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:03.400946 21691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:03.403255 21691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:03.403427 21691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:03.407603 21691 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.124:44623
I20260923 15:09:03.407678 21759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.124:44623 every 8 connection(s)
I20260923 15:09:03.408104 21691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
I20260923 15:09:03.409200 21760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:03.410933 21760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:09:03.412861 21608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } attempt: 2
I20260923 15:09:03.413407 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21691
I20260923 15:09:03.413542 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175983307587-8169-0/minicluster-data/master-2/wal/instance
I20260923 15:09:03.413757 8169 external_mini_cluster.cc:1428] Setting key 4821305210d549f697adf4e41f6fb559
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:09:03.421880 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.411811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55250 (local address 127.7.250.126:44171)
0923 15:09:03.411885 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.411887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.412106 (+ 219us) server_negotiation.cc:407] Connection header received
0923 15:09:03.412609 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.412610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.412620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.412665 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.413562 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.414250 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.415585 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.415885 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.415955 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.418683 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.418684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.418685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.418692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.420163 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.420750 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.420752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.420752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.420817 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.421110 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.421111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.421111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.421317 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.421386 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.421697 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.421785 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:09:03.422015 21612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.413121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49504)
0923 15:09:03.413178 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.413188 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.413226 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.413563 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.413564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.413724 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.413755 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.413756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.415095 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.415097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.415757 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.415760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.416380 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.416381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.416440 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.416788 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.416793 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.418116 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.420617 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.420618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.420618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.420843 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.421192 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.421193 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.421193 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.421248 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.421708 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.421709 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.421754 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.421804 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.421897 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:09:03.422379 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.411778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:44171 (local address 127.0.0.1:55250)
0923 15:09:03.411960 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.412000 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.412513 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.412736 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.412740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.413120 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.413416 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.413422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.414362 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.414363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.415493 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.415496 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.415967 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.415968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.416100 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.416667 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.416676 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.418570 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.420279 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.420287 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.420289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.420657 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.420896 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.420898 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.420898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.421050 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.421419 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.421425 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.421544 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.421662 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.421832 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":89,"thread_start_us":84,"threads_started":1}
I20260923 15:09:03.422518 21763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.413189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49504 (local address 127.7.250.124:44623)
0923 15:09:03.413354 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.413360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.413388 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:09:03.413413 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.413416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.413438 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.413552 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.413854 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.415000 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.415836 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.416283 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.416373 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.418216 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.418225 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.418225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.418233 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.420514 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.420940 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.420942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.420943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.421061 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.421314 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.421315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.421315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.421615 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.422222 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.422293 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.422364 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260923 15:09:03.423074 21760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:09:03.423687 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.414120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49512 (local address 127.7.250.124:44623)
0923 15:09:03.414585 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.414587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.414593 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:09:03.414611 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.414611 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.414624 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.414697 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.415382 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.416038 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.417017 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.417233 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.417319 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.420089 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.420091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.420091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.420097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.422142 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.422562 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.422564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.422564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.422780 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.422992 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.422993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.422993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.423131 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.423285 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.423363 (+ 78us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.423441 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":84,"threads_started":1}
I20260923 15:09:03.424840 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: Bootstrap starting.
I20260923 15:09:03.425590 21608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:03.425920 21608 log.cc:824] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:03.426957 21608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: No bootstrap required, opened a new log
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:09:03.429140 21608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.429303 21608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:03.429354 21608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d11606b8b694f47b3342ccebd9571ef, State: Initialized, Role: FOLLOWER
I20260923 15:09:03.429499 21608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.429843 21608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:03.430408 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.423400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:43769 (local address 127.0.0.1:40308)
0923 15:09:03.423491 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.423502 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.423538 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.423678 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.423679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.423813 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.423868 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.423870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.424508 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.424509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.425241 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.425245 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.425596 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.425597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.425674 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.426300 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.426305 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.427609 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.429243 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.429244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.429245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.429465 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.429711 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.429712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.429712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.429781 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.430065 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.430066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.430116 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.430189 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.430309 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260923 15:09:03.430441 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.423506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40308 (local address 127.7.250.125:43769)
0923 15:09:03.423551 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.423553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.423562 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:09:03.423590 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.423590 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.423600 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.423674 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.423971 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.424423 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.425336 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.425518 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.425567 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.427717 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.427718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.427719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.427726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.429140 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.429561 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.429562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.429562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.429630 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.429854 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.429854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.429855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.429994 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.430072 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.430233 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.430356 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:09:03.430398 21767 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.430483 21767 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:09:03.431005 21760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
W20260923 15:09:03.432562 21778 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:09:03.432653 21778 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:09:03.433341 21540 master_runner.cc:186] Error getting master registration for 127.7.250.125:43769: OK, instance_id { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" instance_seqno: 1790176143236158 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:09:03.436050 21540 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:09:03.438755 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.426901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55254 (local address 127.7.250.126:44171)
0923 15:09:03.426935 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.426937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.426947 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:09:03.426998 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.426999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.427008 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.427058 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.427717 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.428391 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.429351 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.429649 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.429727 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.432950 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.432952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.432952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.432959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.437458 (+ 4499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.437894 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.437896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.437896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.437952 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.438172 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.438173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.438174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.438344 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.438518 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.438567 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.438656 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:09:03.439047 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.431299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49520)
0923 15:09:03.431365 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.431373 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.431418 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.431539 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.431539 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.431672 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.431723 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.431725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.432595 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.432595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.433581 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.433584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.434060 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.434061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.434143 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.434602 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.434607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.436022 (+ 1415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.437842 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.437843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.437843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.438066 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.438334 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.438335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.438335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.438406 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.438765 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.438766 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.438801 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.438860 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.438949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260923 15:09:03.439311 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.431327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49520 (local address 127.7.250.124:44623)
0923 15:09:03.431374 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.431376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.431408 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:09:03.431472 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.431472 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.431482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.431534 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.431785 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.432515 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.433672 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.433964 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.434037 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.436129 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.436130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.436130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.436138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.437753 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.438172 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.438173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.438174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.438225 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.438480 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.438481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.438481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.438678 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.438743 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.439157 (+ 414us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.439233 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:09:03.440881 21760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: Bootstrap starting.
I20260923 15:09:03.441645 21760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:03.441941 21760 log.cc:824] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:03.442801 21760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: No bootstrap required, opened a new log
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:09:03.444868 21760 raft_consensus.cc:348] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.445040 21760 raft_consensus.cc:374] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:03.445115 21760 raft_consensus.cc:729] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 367a97d518604fc79dd01d1d14ae6d85, State: Initialized, Role: FOLLOWER
I20260923 15:09:03.445254 21760 consensus_queue.cc:260] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.445528 21781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.445600 21760 sys_catalog.cc:564] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:03.445663 21781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:09:03.447919 21792 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:09:03.448021 21792 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:09:03.449175 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.439533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.7.250.125:43769)
0923 15:09:03.439576 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.439578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.439629 (+ 51us) server_negotiation.cc:407] Connection header received
0923 15:09:03.439700 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.439701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.439713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.439773 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.440357 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.441099 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.442082 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.442458 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.442526 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.445310 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.445312 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.445312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.445321 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.447145 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.447528 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.447529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.447529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.447584 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.447805 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.447806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.447806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.447946 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.448497 (+ 551us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.449000 (+ 503us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.449080 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:09:03.450469 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:09:03.449018 21691 master_runner.cc:186] Error getting master registration for 127.7.250.125:43769: OK, instance_id { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" instance_seqno: 1790176143236158 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:09:03.451273 21691 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:09:03.460842 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.451876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55258 (local address 127.7.250.126:44171)
0923 15:09:03.451924 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.451926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.452243 (+ 317us) server_negotiation.cc:407] Connection header received
0923 15:09:03.452341 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.452342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.452353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.452382 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.453154 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.453884 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.454982 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.455302 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.455347 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.457276 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.457278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.457278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.457286 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.459574 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.459975 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.459976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.459977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.460032 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.460258 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.460259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.460259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.460387 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.460468 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.460677 (+ 209us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.460754 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:09:03.461365 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.451960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40322 (local address 127.7.250.125:43769)
0923 15:09:03.451993 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.451995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.452476 (+ 481us) server_negotiation.cc:407] Connection header received
0923 15:09:03.452566 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.452567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.452577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.452608 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.453428 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.454164 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.455214 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.455519 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.455559 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.457600 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.457602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.457602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.457608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.459923 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.460288 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.460289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.460289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.460345 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.460533 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.460533 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.460534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.460686 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.460755 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.461190 (+ 435us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.461272 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 15:09:03.467895 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.452257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49532 (local address 127.7.250.124:44623)
0923 15:09:03.452290 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.452292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.453030 (+ 738us) server_negotiation.cc:407] Connection header received
0923 15:09:03.455982 (+ 2952us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.455984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.455994 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.456023 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.457880 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.458635 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.463039 (+ 4404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.463347 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.463419 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.464894 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.464896 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.464896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.464904 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.466668 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.467044 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.467045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.467046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.467110 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.467317 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.467318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.467319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.467440 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.467506 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.467714 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.467796 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:09:03.472584 21684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } attempt: 3
Sep 23 15:09:03 dist-test-slave-bb3x krb5kdc[21527](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176143, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:09:03.480737 21765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.473043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.7.250.124:44623)
0923 15:09:03.473108 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:03.473110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:03.473119 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:09:03.473138 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:03.473139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:03.473149 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:03.473205 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:03.473553 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.474326 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.475299 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.475600 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.475671 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.477811 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:03.477813 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:03.477813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:03.477821 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:03.479487 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.479914 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.479915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.479916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.479976 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:03.480192 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:03.480192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:03.480193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:03.480325 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:03.480393 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:03.480558 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 15:09:03.480642 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:09:03.480727 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:03.472860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44623 (local address 127.0.0.1:49540)
0923 15:09:03.472916 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:03.472927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:03.472961 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:03.473252 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:03.473254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:03.473422 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:03.473473 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.473475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.474408 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.474410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.475213 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:03.475215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:03.475680 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:03.475681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:03.475741 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:03.476206 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:03.476211 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:03.477723 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:03.479582 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.479583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.479583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.479826 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.480045 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:03.480046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:03.480046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:03.480122 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:03.480396 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:03.480397 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:03.480448 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 15:09:03.480522 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:09:03.480625 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:09:03.483515 21684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: Bootstrap starting.
I20260923 15:09:03.484301 21684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:03.484596 21684 log.cc:824] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:03.485589 21684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: No bootstrap required, opened a new log
I20260923 15:09:03.487797 21684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.487941 21684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:03.487974 21684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e402d8e8a34c13b67c615f79c9f95c, State: Initialized, Role: FOLLOWER
I20260923 15:09:03.488191 21684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.488571 21684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:03.488533 21802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.488725 21802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:09:03.493654 21615 master_runner.cc:263] No leader master found from master 91e402d8e8a34c13b67c615f79c9f95c
I20260923 15:09:03.493716 21615 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 15:09:03.494869 21814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:09:03.494931 21814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260923 15:09:03.566197 21561 tablet.cc:2369] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:09:03.654114 21712 tablet.cc:2369] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:09:03.733799 21636 tablet.cc:2369] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:09:03.783061 21767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:03.783197 21767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.783536 21767 leader_election.cc:288] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91e402d8e8a34c13b67c615f79c9f95c (127.7.250.125:43769), 367a97d518604fc79dd01d1d14ae6d85 (127.7.250.124:44623)
I20260923 15:09:03.783919 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d11606b8b694f47b3342ccebd9571ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e402d8e8a34c13b67c615f79c9f95c" is_pre_election: true
I20260923 15:09:03.783942 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d11606b8b694f47b3342ccebd9571ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "367a97d518604fc79dd01d1d14ae6d85" is_pre_election: true
I20260923 15:09:03.784096 21658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d11606b8b694f47b3342ccebd9571ef in term 0.
I20260923 15:09:03.784116 21734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d11606b8b694f47b3342ccebd9571ef in term 0.
I20260923 15:09:03.784303 21557 leader_election.cc:302] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e402d8e8a34c13b67c615f79c9f95c, 9d11606b8b694f47b3342ccebd9571ef; no voters:
I20260923 15:09:03.784511 21767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:09:03.784591 21767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:09:03.784608 21767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:03.785458 21767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.785571 21767 leader_election.cc:288] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [CANDIDATE]: Term 1 election: Requested vote from peers 91e402d8e8a34c13b67c615f79c9f95c (127.7.250.125:43769), 367a97d518604fc79dd01d1d14ae6d85 (127.7.250.124:44623)
I20260923 15:09:03.785799 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d11606b8b694f47b3342ccebd9571ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91e402d8e8a34c13b67c615f79c9f95c"
I20260923 15:09:03.785849 21734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d11606b8b694f47b3342ccebd9571ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "367a97d518604fc79dd01d1d14ae6d85"
I20260923 15:09:03.785900 21658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:03.785938 21734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:03.786875 21734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d11606b8b694f47b3342ccebd9571ef in term 1.
I20260923 15:09:03.786887 21658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d11606b8b694f47b3342ccebd9571ef in term 1.
I20260923 15:09:03.787087 21557 leader_election.cc:302] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91e402d8e8a34c13b67c615f79c9f95c, 9d11606b8b694f47b3342ccebd9571ef; no voters:
I20260923 15:09:03.787359 21767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:03.787488 21767 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [term 1 LEADER]: Becoming Leader. State: Replica: 9d11606b8b694f47b3342ccebd9571ef, State: Running, Role: LEADER
I20260923 15:09:03.787632 21767 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } }
I20260923 15:09:03.788084 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d11606b8b694f47b3342ccebd9571ef. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.788239 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:03.788465 21818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:03.788710 21818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:09:03.790234 21658 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [term 1 FOLLOWER]: Refusing update from remote peer 9d11606b8b694f47b3342ccebd9571ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:03.790300 21734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [term 1 FOLLOWER]: Refusing update from remote peer 9d11606b8b694f47b3342ccebd9571ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:03.790627 21767 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:03.790755 21816 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:03.792061 21781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d11606b8b694f47b3342ccebd9571ef. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.792160 21781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:09:03.792060 21802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d11606b8b694f47b3342ccebd9571ef. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.792194 21802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:09:03.793097 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.793186 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:03.793231 21816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.793267 21816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:03.793303 21819 mvcc.cc:205] Tried to move back new op lower bound from 7332561484962123776 to 7332561484954812416. Current Snapshot: MvccSnapshot[applied={T|T < 7332561484962123776}]
I20260923 15:09:03.793309 21821 mvcc.cc:205] Tried to move back new op lower bound from 7332561484962123776 to 7332561484954812416. Current Snapshot: MvccSnapshot[applied={T|T < 7332561484962123776}]
I20260923 15:09:03.793373 21781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.793457 21781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:09:03.793516 21818 catalog_manager.cc:1347] Generated new cluster ID: d00e5065b2c14ab4b6c579403aeb847d
I20260923 15:09:03.793529 21802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d11606b8b694f47b3342ccebd9571ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9d11606b8b694f47b3342ccebd9571ef" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44171 } } peers { permanent_uuid: "91e402d8e8a34c13b67c615f79c9f95c" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 43769 } } peers { permanent_uuid: "367a97d518604fc79dd01d1d14ae6d85" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44623 } } }
I20260923 15:09:03.793583 21818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:09:03.793606 21802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:09:03.793924 21820 mvcc.cc:205] Tried to move back new op lower bound from 7332561484962123776 to 7332561484954812416. Current Snapshot: MvccSnapshot[applied={T|T < 7332561484962123776}]
I20260923 15:09:03.809955 21818 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:09:03.810653 21818 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:09:03.817441 21818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d11606b8b694f47b3342ccebd9571ef: Generated new TSK 0
I20260923 15:09:03.817654 21818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:09:04.448594 21792 catalog_manager.cc:1259] Loaded cluster ID: d00e5065b2c14ab4b6c579403aeb847d
I20260923 15:09:04.448671 21792 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: loading cluster ID for follower catalog manager: success
I20260923 15:09:04.450028 21792 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: acquiring CA information for follower catalog manager: success
I20260923 15:09:04.450387 21792 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 367a97d518604fc79dd01d1d14ae6d85: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:09:04.495404 21814 catalog_manager.cc:1259] Loaded cluster ID: d00e5065b2c14ab4b6c579403aeb847d
I20260923 15:09:04.495512 21814 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: loading cluster ID for follower catalog manager: success
I20260923 15:09:04.496416 21814 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: acquiring CA information for follower catalog manager: success
I20260923 15:09:04.496737 21814 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 91e402d8e8a34c13b67c615f79c9f95c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:09:04.871987 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21540
I20260923 15:09:04.877224 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21615
I20260923 15:09:04.882405 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21691
2026-09-23T15:09:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3161 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:09:04 dist-test-slave-bb3x krb5kdc[21833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:09:04 dist-test-slave-bb3x krb5kdc[21833](info): set up 2 sockets
Sep 23 15:09:04 dist-test-slave-bb3x krb5kdc[21833](info): commencing operation
krb5kdc: starting...
W20260923 15:09:05.971967 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:09:06 dist-test-slave-bb3x krb5kdc[21833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:09:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:09:06.098363 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34589
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:34589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:06.182106 21846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:06.182374 21846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:06.182402 21846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:06.183585 21846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:06.183646 21846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:06.183663 21846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:06.183677 21846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:06.183691 21846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:06.185163 21846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:34589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:06.185391 21846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:06.185585 21846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:06.191180 21852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:06.191246 21851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:06.191183 21854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:06.191341 21846 server_base.cc:1034] running on GCE node
I20260923 15:09:06.191639 21846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:06.191985 21846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:06.193173 21846 hybrid_clock.cc:648] HybridClock initialized: now 1790176146193145 us; error 44 us; skew 500 ppm
Sep 23 15:09:06 dist-test-slave-bb3x krb5kdc[21833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176146, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:06.195930 21846 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:06.196424 21846 webserver.cc:460] Webserver started at http://127.7.250.126:33799/ using document root <none> and password file <none>
I20260923 15:09:06.196682 21846 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:06.196754 21846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:06.196903 21846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:06.197976 21846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "a2d67462ad5b42008b7630dec53f2d6b"
format_stamp: "Formatted at 2026-09-23 15:09:06 on dist-test-slave-bb3x"
server_key: "8ba6e3fca22cc825cc3de8d647ba9adf"
server_key_iv: "b846ff1f11c88bdca24aa47c5db453c0"
server_key_version: "encryptionkey@0"
I20260923 15:09:06.198400 21846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "a2d67462ad5b42008b7630dec53f2d6b"
format_stamp: "Formatted at 2026-09-23 15:09:06 on dist-test-slave-bb3x"
server_key: "8ba6e3fca22cc825cc3de8d647ba9adf"
server_key_iv: "b846ff1f11c88bdca24aa47c5db453c0"
server_key_version: "encryptionkey@0"
I20260923 15:09:06.199776 21846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:06.200646 21861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:06.200877 21846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:06.201035 21846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "a2d67462ad5b42008b7630dec53f2d6b"
format_stamp: "Formatted at 2026-09-23 15:09:06 on dist-test-slave-bb3x"
server_key: "8ba6e3fca22cc825cc3de8d647ba9adf"
server_key_iv: "b846ff1f11c88bdca24aa47c5db453c0"
server_key_version: "encryptionkey@0"
I20260923 15:09:06.201122 21846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:06.211889 21846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:06.214367 21846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:06.214540 21846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:06.218603 21846 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:34589
I20260923 15:09:06.218679 21913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:34589 every 8 connection(s)
I20260923 15:09:06.219041 21846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:06.219971 21914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:06.222319 21914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b: Bootstrap starting.
I20260923 15:09:06.222918 21914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:06.223232 21914 log.cc:824] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:06.223730 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21846
I20260923 15:09:06.223840 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:09:06.224063 8169 external_mini_cluster.cc:1428] Setting key a18cc9d68806e20fe617c2fc6d90b0f5
I20260923 15:09:06.224182 21914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b: No bootstrap required, opened a new log
I20260923 15:09:06.226794 21914 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d67462ad5b42008b7630dec53f2d6b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34589 } }
I20260923 15:09:06.227010 21914 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:06.227058 21914 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2d67462ad5b42008b7630dec53f2d6b, State: Initialized, Role: FOLLOWER
I20260923 15:09:06.227208 21914 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d67462ad5b42008b7630dec53f2d6b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34589 } }
I20260923 15:09:06.227290 21914 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:09:06.227324 21914 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:09:06.227371 21914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:06.228150 21914 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d67462ad5b42008b7630dec53f2d6b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34589 } }
I20260923 15:09:06.228267 21914 leader_election.cc:302] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2d67462ad5b42008b7630dec53f2d6b; no voters:
I20260923 15:09:06.228447 21914 leader_election.cc:288] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:09:06.228505 21919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:06.228680 21919 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [term 1 LEADER]: Becoming Leader. State: Replica: a2d67462ad5b42008b7630dec53f2d6b, State: Running, Role: LEADER
I20260923 15:09:06.228722 21914 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:06.228771 21919 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d67462ad5b42008b7630dec53f2d6b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34589 } }
I20260923 15:09:06.229152 21920 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2d67462ad5b42008b7630dec53f2d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d67462ad5b42008b7630dec53f2d6b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34589 } } }
I20260923 15:09:06.229292 21920 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:06.229143 21919 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2d67462ad5b42008b7630dec53f2d6b. Latest consensus state: current_term: 1 leader_uuid: "a2d67462ad5b42008b7630dec53f2d6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2d67462ad5b42008b7630dec53f2d6b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34589 } } }
I20260923 15:09:06.229400 21919 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:06.230113 21927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:06.230623 21927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:09:06 dist-test-slave-bb3x krb5kdc[21833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:06.233182 21927 catalog_manager.cc:1347] Generated new cluster ID: cf196645f26c411aa4c2bfb577ea44ed
I20260923 15:09:06.233275 21927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:09:06.238356 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:06.224558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.7.250.126:34589)
0923 15:09:06.224749 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:06.224760 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:06.224808 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:09:06.225748 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:06.225759 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:06.225829 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:06.226007 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:06.226675 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:06.228602 (+ 1927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:06.229939 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:06.230781 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:06.230901 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:06.233476 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:06.233481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:06.233481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:06.233494 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:06.235788 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:06.236215 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:06.236219 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:06.236220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:06.236344 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:06.236574 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:06.236575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:06.236576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:06.237061 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:06.237241 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:06.237904 (+ 663us) server_negotiation.cc:299] Negotiation successful
0923 15:09:06.238087 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260923 15:09:06.239557 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:09:06.250844 21927 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:09:06.251519 21927 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:09:06.258633 21927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2d67462ad5b42008b7630dec53f2d6b: Generated new TSK 0
I20260923 15:09:06.258837 21927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:09:06.293821 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21846
2026-09-23T15:09:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1409 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:09:06 dist-test-slave-bb3x krb5kdc[21941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:09:06 dist-test-slave-bb3x krb5kdc[21941](info): set up 2 sockets
Sep 23 15:09:06 dist-test-slave-bb3x krb5kdc[21941](info): commencing operation
krb5kdc: starting...
W20260923 15:09:07.396010 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:09:07 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:09:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:09:07.522198 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34921
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:07.602741 21954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:07.602989 21954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:07.603029 21954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:07.604261 21954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:07.604338 21954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:07.604363 21954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:07.604399 21954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:07.604429 21954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:07.605890 21954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:34921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:07.606166 21954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:07.606457 21954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:07.611502 21962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:07.611527 21961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:07.611528 21964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:07.611578 21954 server_base.cc:1034] running on GCE node
I20260923 15:09:07.611937 21954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:07.612272 21954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:07.613471 21954 hybrid_clock.cc:648] HybridClock initialized: now 1790176147613383 us; error 102 us; skew 500 ppm
Sep 23 15:09:07 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:07.616091 21954 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:07.616596 21954 webserver.cc:460] Webserver started at http://127.7.250.126:37131/ using document root <none> and password file <none>
I20260923 15:09:07.616849 21954 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:07.616920 21954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:07.617070 21954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:07.618140 21954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "b20f7ba48020487a8a1937341160251f"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "b2151ce069a91fa162094ea12fe625c1"
server_key_iv: "ef7b452d6d102728163dee32f973ed86"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.618635 21954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "b20f7ba48020487a8a1937341160251f"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "b2151ce069a91fa162094ea12fe625c1"
server_key_iv: "ef7b452d6d102728163dee32f973ed86"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.619938 21954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:07.620645 21971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:07.620824 21954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:09:07.620921 21954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "b20f7ba48020487a8a1937341160251f"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "b2151ce069a91fa162094ea12fe625c1"
server_key_iv: "ef7b452d6d102728163dee32f973ed86"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.621011 21954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:07.639765 21954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:07.642328 21954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:07.642519 21954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:07.646624 21954 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:34921
I20260923 15:09:07.646677 22023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:34921 every 8 connection(s)
I20260923 15:09:07.647050 21954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:07.647187 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 21954
I20260923 15:09:07.647315 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:09:07.647555 8169 external_mini_cluster.cc:1428] Setting key 983f36ca4383358b4823648b05cc0feb
I20260923 15:09:07.648270 22024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:07.650748 22024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap starting.
I20260923 15:09:07.651373 22024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:07.651692 22024 log.cc:824] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:07.652531 22024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: No bootstrap required, opened a new log
Sep 23 15:09:07 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:07.655241 22024 raft_consensus.cc:348] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:07.655441 22024 raft_consensus.cc:374] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:07.655494 22024 raft_consensus.cc:729] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b20f7ba48020487a8a1937341160251f, State: Initialized, Role: FOLLOWER
I20260923 15:09:07.655702 22024 consensus_queue.cc:260] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:07.655807 22024 raft_consensus.cc:388] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:09:07.655848 22024 raft_consensus.cc:482] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:09:07.655893 22024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:07.656692 22024 raft_consensus.cc:504] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:07.656801 22024 leader_election.cc:302] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b20f7ba48020487a8a1937341160251f; no voters:
I20260923 15:09:07.656980 22024 leader_election.cc:288] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:09:07.657078 22029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:07.657266 22024 sys_catalog.cc:564] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:07.657310 22029 raft_consensus.cc:686] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 LEADER]: Becoming Leader. State: Replica: b20f7ba48020487a8a1937341160251f, State: Running, Role: LEADER
I20260923 15:09:07.657509 22029 consensus_queue.cc:237] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:07.657941 22030 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b20f7ba48020487a8a1937341160251f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } } }
I20260923 15:09:07.658030 22030 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:07.658221 22038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:07.658241 22031 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b20f7ba48020487a8a1937341160251f. Latest consensus state: current_term: 1 leader_uuid: "b20f7ba48020487a8a1937341160251f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } } }
I20260923 15:09:07.658396 22031 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:07.658797 22038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:09:07.659523 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:07.648083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38448 (local address 127.7.250.126:34921)
0923 15:09:07.648278 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:07.648288 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:07.648320 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:09:07.648826 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:07.648833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:07.648879 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:07.649024 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:07.649683 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:07.650656 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:07.651507 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:07.652019 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:07.652126 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:07.655035 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:07.655040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:07.655040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:07.655054 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:07.657246 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:07.657719 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:07.657722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:07.657722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:07.657846 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:07.658124 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:07.658126 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:07.658126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:07.658516 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:07.658688 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:07.659164 (+ 476us) server_negotiation.cc:299] Negotiation successful
0923 15:09:07.659307 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":87,"threads_started":1}
I20260923 15:09:07.660632 22038 catalog_manager.cc:1347] Generated new cluster ID: 7d199e4a4bab4a15bbc3a55fb573085f
I20260923 15:09:07.660682 22038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:09:07.673727 22038 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:09:07.674345 22038 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:09:07.677300 22038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Generated new TSK 0
I20260923 15:09:07.677500 22038 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:09:07.711374 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34921
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:07.806193 22050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:07.806453 22050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:07.806483 22050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:07.807827 22050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:07.807871 22050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:07.807910 22050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:09:07.809468 22050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:07.809697 22050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:07.809904 22050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:07.815176 22056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:07.815281 22055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:07.815330 22058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:07.815541 22050 server_base.cc:1034] running on GCE node
I20260923 15:09:07.815732 22050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:07.815958 22050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:07.817132 22050 hybrid_clock.cc:648] HybridClock initialized: now 1790176147817117 us; error 29 us; skew 500 ppm
Sep 23 15:09:07 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:07.819954 22050 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:09:07.820408 22050 webserver.cc:460] Webserver started at http://127.7.250.65:44853/ using document root <none> and password file <none>
I20260923 15:09:07.820663 22050 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:07.820736 22050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:07.820884 22050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:07.822027 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "7d6494680a8b4118931c3b450db242ce"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "c0b86714e6c21a19b8eabf17d80b8428"
server_key_iv: "8b66f650850b9ae8b8ab46097dc9a9dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.822417 22050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "7d6494680a8b4118931c3b450db242ce"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "c0b86714e6c21a19b8eabf17d80b8428"
server_key_iv: "8b66f650850b9ae8b8ab46097dc9a9dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.823871 22050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:07.824805 22065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:07.825044 22050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:07.825147 22050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "7d6494680a8b4118931c3b450db242ce"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "c0b86714e6c21a19b8eabf17d80b8428"
server_key_iv: "8b66f650850b9ae8b8ab46097dc9a9dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.825228 22050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:07.836402 22050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:07.838769 22050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:07.838944 22050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:07.839178 22050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:07.839489 22050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:07.839543 22050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:07.839586 22050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:07.839612 22050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:07.845733 22050 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43637
I20260923 15:09:07.845803 22179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43637 every 8 connection(s)
I20260923 15:09:07.846151 22050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:09:07.847365 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22050
I20260923 15:09:07.847482 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:09:07.847661 8169 external_mini_cluster.cc:1428] Setting key ea924d3ecce8303392c0953df221ae02
Sep 23 15:09:07 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:07.857523 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:07.847104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:59823 (local address 127.7.250.126:34921)
0923 15:09:07.847177 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:07.847180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:07.847282 (+ 102us) server_negotiation.cc:407] Connection header received
0923 15:09:07.848023 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:07.848024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:07.848036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:07.848093 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:07.849298 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:07.849855 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:07.851089 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:07.851447 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:07.851525 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:07.854163 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:07.854165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:07.854165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:07.854175 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:07.855969 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:07.856374 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:07.856375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:07.856376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:07.856444 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:07.856703 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:07.856704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:07.856704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:07.856872 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:07.856972 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:07.857292 (+ 320us) server_negotiation.cc:299] Negotiation successful
0923 15:09:07.857388 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:09:07.857650 22182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:07.846901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:59823)
0923 15:09:07.847111 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:07.847197 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:07.847910 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:07.848171 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:07.848177 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:07.848940 (+ 763us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:07.849166 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:07.849172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:07.849975 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:07.849976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:07.850980 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:07.850983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:07.851542 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:07.851543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:07.851654 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:07.852252 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:07.852261 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:07.854057 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:07.856071 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:07.856078 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:07.856080 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:07.856292 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:07.856524 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:07.856524 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:07.856524 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:07.856625 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:07.856972 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:07.856978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:07.857140 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 15:09:07.857231 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:09:07.857355 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":66,"threads_started":1}
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
I20260923 15:09:07.858299 22180 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34921
I20260923 15:09:07.858444 22180 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:07.858646 22180 heartbeater.cc:511] Master 127.7.250.126:34921 requested a full tablet report, sending...
I20260923 15:09:07.859275 21988 ts_manager.cc:194] Registered new tserver with Master: 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637)
I20260923 15:09:07.859921 21988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:59823
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:09:07.877971 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34921
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:07.962337 22185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:07.962589 22185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:07.962630 22185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:07.963838 22185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:07.963912 22185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:07.963960 22185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:09:07.965492 22185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:07.965775 22185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:07.966024 22185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:07.971837 22191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:07.971913 22185 server_base.cc:1034] running on GCE node
W20260923 15:09:07.971925 22190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:07.972067 22193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:07.972306 22185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:07.972616 22185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:07.973793 22185 hybrid_clock.cc:648] HybridClock initialized: now 1790176147973768 us; error 40 us; skew 500 ppm
Sep 23 15:09:07 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:07.976481 22185 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:09:07.977020 22185 webserver.cc:460] Webserver started at http://127.7.250.66:36871/ using document root <none> and password file <none>
I20260923 15:09:07.977269 22185 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:07.977340 22185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:07.977496 22185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:07.978655 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "8f904add515d0f637066cf03f4afc9e2"
server_key_iv: "a09877fce13f9bde325ebc80565ac07a"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.979028 22185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "8f904add515d0f637066cf03f4afc9e2"
server_key_iv: "a09877fce13f9bde325ebc80565ac07a"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.980460 22185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:07.981271 22200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:07.981565 22185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:07.981689 22185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "8f904add515d0f637066cf03f4afc9e2"
server_key_iv: "a09877fce13f9bde325ebc80565ac07a"
server_key_version: "encryptionkey@0"
I20260923 15:09:07.981769 22185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:07.991897 22185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:07.994148 22185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:07.994361 22185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:07.994621 22185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:07.994951 22185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:07.995014 22185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:07.995059 22185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:07.995083 22185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.001420 22185 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:41761
I20260923 15:09:08.001492 22314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:41761 every 8 connection(s)
I20260923 15:09:08.001798 22185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:09:08.003135 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22185
I20260923 15:09:08.003257 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:09:08.003484 8169 external_mini_cluster.cc:1428] Setting key a5ba60f77b7725495a4ce529de85e3c8
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176147, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
I20260923 15:09:08.013715 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.002627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:32861 (local address 127.7.250.126:34921)
0923 15:09:08.002721 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.002724 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.003010 (+ 286us) server_negotiation.cc:407] Connection header received
0923 15:09:08.003828 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.003829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.003842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.003918 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:08.005083 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.005859 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.007003 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.007306 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.007418 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.010270 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:08.010272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:08.010273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:08.010284 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:08.012008 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.012546 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.012550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.012551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.012614 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.012919 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.012920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.012920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.013087 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:08.013174 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.013499 (+ 325us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.013609 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:09:08.013962 22317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.002535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:32861)
0923 15:09:08.002778 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.002836 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.003703 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.004002 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.004007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.004754 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:08.004975 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.004980 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.005972 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.005973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.006894 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.006897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.007432 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.007434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.007594 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.008256 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:08.008265 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:08.010126 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:08.012130 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.012139 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.012143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.012424 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.012703 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.012704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.012705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.012832 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.013163 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:08.013171 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:08.013342 (+ 171us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.013469 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.013669 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260923 15:09:08.014596 22315 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34921
I20260923 15:09:08.014704 22315 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:08.014859 22315 heartbeater.cc:511] Master 127.7.250.126:34921 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
I20260923 15:09:08.015254 21988 ts_manager.cc:194] Registered new tserver with Master: bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761)
I20260923 15:09:08.015677 21988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:32861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:09:08.031337 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34921
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:08.117182 22320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:08.117440 22320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:08.117486 22320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:08.118688 22320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:08.118764 22320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:08.118810 22320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:09:08.120437 22320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:08.120712 22320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:08.120956 22320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:08.126288 22326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:08.126297 22325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.126389 22320 server_base.cc:1034] running on GCE node
W20260923 15:09:08.126288 22328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.126722 22320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:08.126976 22320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:08.128260 22320 hybrid_clock.cc:648] HybridClock initialized: now 1790176148128250 us; error 30 us; skew 500 ppm
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:08.135804 22320 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:09:08.136372 22320 webserver.cc:460] Webserver started at http://127.7.250.67:39395/ using document root <none> and password file <none>
I20260923 15:09:08.136631 22320 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:08.136678 22320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:08.136806 22320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:08.137926 22320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "e10ec0d212df4bcf8bdf401861092926"
format_stamp: "Formatted at 2026-09-23 15:09:08 on dist-test-slave-bb3x"
server_key: "590de1a6454fb071c5bfc4f5c40b3690"
server_key_iv: "607f575a795d2e03bda887b4e894b0dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:08.138353 22320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "e10ec0d212df4bcf8bdf401861092926"
format_stamp: "Formatted at 2026-09-23 15:09:08 on dist-test-slave-bb3x"
server_key: "590de1a6454fb071c5bfc4f5c40b3690"
server_key_iv: "607f575a795d2e03bda887b4e894b0dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:08.139801 22320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:08.140887 22335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.141124 22320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:08.141224 22320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "e10ec0d212df4bcf8bdf401861092926"
format_stamp: "Formatted at 2026-09-23 15:09:08 on dist-test-slave-bb3x"
server_key: "590de1a6454fb071c5bfc4f5c40b3690"
server_key_iv: "607f575a795d2e03bda887b4e894b0dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:08.141290 22320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:08.158598 22320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:08.161300 22320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:08.161429 22320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:08.161651 22320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:08.161935 22320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:08.161970 22320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.161998 22320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:08.162014 22320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.168663 22320 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:38253
I20260923 15:09:08.168740 22448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:38253 every 8 connection(s)
I20260923 15:09:08.169097 22320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:08.177023 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22320
I20260923 15:09:08.177152 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:09:08.177397 8169 external_mini_cluster.cc:1428] Setting key 7327cb8c6f659a5bef95eedfee211cba
I20260923 15:09:08.180284 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.170015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:41655 (local address 127.7.250.126:34921)
0923 15:09:08.170099 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.170101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.170309 (+ 208us) server_negotiation.cc:407] Connection header received
0923 15:09:08.170888 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.170889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.170902 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.170949 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:08.172048 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.172556 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.173606 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.173806 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.173906 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.176421 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:08.176423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:08.176423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:08.176436 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:08.178591 (+ 2155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.179118 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.179119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.179119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.179202 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.179502 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.179503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.179504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.179652 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:08.179747 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.180075 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.180173 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:09:08.180550 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.169867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:41655)
0923 15:09:08.170136 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.170195 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.170771 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.171102 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.171106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.171710 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:08.171917 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.171922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.172660 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.172661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.173486 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.173488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.173934 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.173935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.174065 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.174673 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:08.174681 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:08.176262 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:08.178703 (+ 2441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.178713 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.178716 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.179009 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.179283 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.179285 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.179285 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.179419 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.179735 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:08.179743 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:08.179919 (+ 176us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.180050 (+ 131us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.180241 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":103,"threads_started":1}
I20260923 15:09:08.181066 22449 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34921
I20260923 15:09:08.181164 22449 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:08.181346 22449 heartbeater.cc:511] Master 127.7.250.126:34921 requested a full tablet report, sending...
I20260923 15:09:08.181733 21988 ts_manager.cc:194] Registered new tserver with Master: e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
I20260923 15:09:08.182117 21988 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:41655
I20260923 15:09:08.189827 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:09:08.198094 22027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.191591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.7.250.126:34921)
0923 15:09:08.191661 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.191663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.191868 (+ 205us) server_negotiation.cc:407] Connection header received
0923 15:09:08.191895 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.191896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.191907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.191968 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:08.192623 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.193107 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.193810 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.193993 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.194085 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.195414 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:08.195415 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:08.195416 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:08.195422 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:08.196889 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.197238 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.197239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.197239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.197298 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.197468 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.197469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.197469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.197621 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:08.197717 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.197810 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.197934 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:09:08.199982 21988 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:09:08.201635 21988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:09:08.208680 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.204571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53034 (local address 127.7.250.67:38253)
0923 15:09:08.204769 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.204773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.204784 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:09:08.204885 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.204888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.204904 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.204982 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:08.205212 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.205975 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.207589 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.208232 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.208335 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.208402 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.208547 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.208590 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":72,"threads_started":1}
I20260923 15:09:08.208674 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.204326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:38253 (local address 127.0.0.1:53034)
0923 15:09:08.204728 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.204739 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.204795 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.205070 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.205071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.205073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:08.205132 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.205133 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.206083 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.206084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.207490 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.207493 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.208338 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.208339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.208418 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.208433 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.208530 (+ 97us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.208591 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":104,"threads_started":1}
I20260923 15:09:08.208850 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.204319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43637 (local address 127.0.0.1:32936)
0923 15:09:08.204492 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.204533 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.204603 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.205189 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.205190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.205193 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:08.205246 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.205247 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.206544 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.206545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.207471 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.207474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.208508 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.208509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.208592 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.208596 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.208643 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.208687 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":50,"threads_started":1}
I20260923 15:09:08.208914 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.204500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32936 (local address 127.7.250.65:43637)
0923 15:09:08.204948 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.204952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.204963 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:09:08.205000 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.205002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.205017 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.205099 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:08.205343 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.206441 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.207556 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.208330 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.208403 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.208469 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.208741 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.208794 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":93,"threads_started":1}
I20260923 15:09:08.209003 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.204326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41761 (local address 127.0.0.1:52558)
0923 15:09:08.204714 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.204723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.204762 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.204964 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.204968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.204971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:08.205071 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.205074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.206517 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.206518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.207639 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.207641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.208674 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.208675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.208729 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.208734 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.208884 (+ 150us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.208925 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":27,"thread_start_us":104,"threads_started":1}
I20260923 15:09:08.209218 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.204581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.7.250.66:41761)
0923 15:09:08.204748 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.204753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.204764 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:09:08.204827 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.204829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.204846 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.204929 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:08.205157 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.206410 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.207723 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.208520 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.208582 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.208631 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.209110 (+ 479us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.209153 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260923 15:09:08.209908 22248 tablet_service.cc:1467] Processing CreateTablet for tablet 479d17af68ee4176acd3fe06775cbf15 (DEFAULT_TABLE table=test-table [id=35569c596be94b868ab2eefaae9a9ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:08.209903 22114 tablet_service.cc:1467] Processing CreateTablet for tablet 479d17af68ee4176acd3fe06775cbf15 (DEFAULT_TABLE table=test-table [id=35569c596be94b868ab2eefaae9a9ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:08.209998 22383 tablet_service.cc:1467] Processing CreateTablet for tablet 479d17af68ee4176acd3fe06775cbf15 (DEFAULT_TABLE table=test-table [id=35569c596be94b868ab2eefaae9a9ca5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:08.210301 22114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479d17af68ee4176acd3fe06775cbf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:08.210300 22383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479d17af68ee4176acd3fe06775cbf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:08.210328 22248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 479d17af68ee4176acd3fe06775cbf15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:08.213043 22466 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Bootstrap starting.
I20260923 15:09:08.213595 22467 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Bootstrap starting.
I20260923 15:09:08.213949 22466 tablet_bootstrap.cc:654] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:08.214244 22467 tablet_bootstrap.cc:654] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:08.214462 22468 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Bootstrap starting.
I20260923 15:09:08.214807 22467 log.cc:824] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:08.214844 22466 log.cc:824] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:08.215188 22468 tablet_bootstrap.cc:654] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:08.215582 22466 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: No bootstrap required, opened a new log
I20260923 15:09:08.215590 22467 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: No bootstrap required, opened a new log
I20260923 15:09:08.215657 22466 ts_tablet_manager.cc:1397] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:09:08.215657 22467 ts_tablet_manager.cc:1397] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:09:08.215703 22468 log.cc:824] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:08.216374 22468 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: No bootstrap required, opened a new log
I20260923 15:09:08.216454 22468 ts_tablet_manager.cc:1397] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:09:08.217626 22467 raft_consensus.cc:348] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.217623 22466 raft_consensus.cc:348] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.217810 22466 raft_consensus.cc:374] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:08.217810 22467 raft_consensus.cc:374] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:08.217849 22466 raft_consensus.cc:729] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf1565cf61e4f9b9ad8c4de288574af, State: Initialized, Role: FOLLOWER
I20260923 15:09:08.217859 22467 raft_consensus.cc:729] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10ec0d212df4bcf8bdf401861092926, State: Initialized, Role: FOLLOWER
I20260923 15:09:08.217801 22468 raft_consensus.cc:348] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.217976 22466 consensus_queue.cc:260] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.218011 22468 raft_consensus.cc:374] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:08.218003 22467 consensus_queue.cc:260] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.218053 22468 raft_consensus.cc:729] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6494680a8b4118931c3b450db242ce, State: Initialized, Role: FOLLOWER
I20260923 15:09:08.218190 22468 consensus_queue.cc:260] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.218288 22467 ts_tablet_manager.cc:1428] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:08.218292 22466 ts_tablet_manager.cc:1428] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:08.218423 22449 heartbeater.cc:503] Master 127.7.250.126:34921 was elected leader, sending a full tablet report...
I20260923 15:09:08.218505 22468 ts_tablet_manager.cc:1428] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:09:08.218609 22180 heartbeater.cc:503] Master 127.7.250.126:34921 was elected leader, sending a full tablet report...
I20260923 15:09:08.218605 22315 heartbeater.cc:503] Master 127.7.250.126:34921 was elected leader, sending a full tablet report...
W20260923 15:09:08.252395 22316 tablet.cc:2369] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:09:08.347028 22181 tablet.cc:2369] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:09:08.419739 22450 tablet.cc:2369] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:09:08.547493 22473 raft_consensus.cc:482] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:08.547636 22473 raft_consensus.cc:504] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.547973 22473 leader_election.cc:288] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637)
I20260923 15:09:08.551015 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.548187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43637 (local address 127.7.250.67:59707)
0923 15:09:08.548247 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.548261 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.548313 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.548542 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.548543 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.548545 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:08.548620 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.548622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.549292 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.549293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.550156 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.550159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.550746 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.550747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.550829 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.550832 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.550895 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.550939 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:09:08.551016 22463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.548352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:59707 (local address 127.7.250.65:43637)
0923 15:09:08.548426 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.548429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.548436 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:09:08.548460 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.548461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.548468 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.548533 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:08.548697 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.549207 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.550283 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.550655 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.550743 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.550752 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.550914 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.550956 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:09:08.551357 22465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.548273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:48251 (local address 127.7.250.66:41761)
0923 15:09:08.548331 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.548333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.548590 (+ 257us) server_negotiation.cc:407] Connection header received
0923 15:09:08.548664 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.548665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.548676 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.548751 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:08.548936 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.549449 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.550540 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.550933 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.551019 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.551028 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.551241 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.551286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:09:08.551383 22134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479d17af68ee4176acd3fe06775cbf15" candidate_uuid: "e10ec0d212df4bcf8bdf401861092926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6494680a8b4118931c3b450db242ce" is_pre_election: true
I20260923 15:09:08.551556 22134 raft_consensus.cc:2454] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10ec0d212df4bcf8bdf401861092926 in term 0.
I20260923 15:09:08.551592 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.548204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41761 (local address 127.7.250.67:48251)
0923 15:09:08.548492 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.548503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.548597 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.548770 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.548771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.548772 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:08.548857 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.548858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.549551 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.549553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.550444 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.550446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.551036 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.551037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.551120 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.551126 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.551461 (+ 335us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.551510 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":98,"thread_start_us":116,"threads_started":1}
I20260923 15:09:08.551816 22337 leader_election.cc:302] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce, e10ec0d212df4bcf8bdf401861092926; no voters:
I20260923 15:09:08.552007 22473 raft_consensus.cc:2781] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:09:08.552071 22473 raft_consensus.cc:482] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:09:08.552104 22473 raft_consensus.cc:3037] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:08.552002 22269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479d17af68ee4176acd3fe06775cbf15" candidate_uuid: "e10ec0d212df4bcf8bdf401861092926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" is_pre_election: true
I20260923 15:09:08.552196 22269 raft_consensus.cc:2454] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e10ec0d212df4bcf8bdf401861092926 in term 0.
I20260923 15:09:08.552960 22473 raft_consensus.cc:504] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.553124 22473 leader_election.cc:288] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [CANDIDATE]: Term 1 election: Requested vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637)
I20260923 15:09:08.553278 22269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479d17af68ee4176acd3fe06775cbf15" candidate_uuid: "e10ec0d212df4bcf8bdf401861092926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
I20260923 15:09:08.553303 22134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "479d17af68ee4176acd3fe06775cbf15" candidate_uuid: "e10ec0d212df4bcf8bdf401861092926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d6494680a8b4118931c3b450db242ce"
I20260923 15:09:08.553375 22269 raft_consensus.cc:3037] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:08.553377 22134 raft_consensus.cc:3037] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:08.554095 22269 raft_consensus.cc:2454] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10ec0d212df4bcf8bdf401861092926 in term 1.
I20260923 15:09:08.554234 22134 raft_consensus.cc:2454] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e10ec0d212df4bcf8bdf401861092926 in term 1.
I20260923 15:09:08.554332 22338 leader_election.cc:302] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdf1565cf61e4f9b9ad8c4de288574af, e10ec0d212df4bcf8bdf401861092926; no voters:
I20260923 15:09:08.554422 22473 raft_consensus.cc:2781] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:08.554587 22473 raft_consensus.cc:686] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 1 LEADER]: Becoming Leader. State: Replica: e10ec0d212df4bcf8bdf401861092926, State: Running, Role: LEADER
I20260923 15:09:08.554694 22473 consensus_queue.cc:237] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.555543 21988 catalog_manager.cc:5495] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 reported cstate change: term changed from 0 to 1, leader changed from <none> to e10ec0d212df4bcf8bdf401861092926 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "e10ec0d212df4bcf8bdf401861092926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } health_report { overall_health: HEALTHY } } }
I20260923 15:09:08.564502 22464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.562140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53042 (local address 127.7.250.67:38253)
0923 15:09:08.562196 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.562198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.562206 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:09:08.562334 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.562335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.562350 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.562409 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:08.562546 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.562992 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.563776 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.563954 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.564007 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.564159 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:08.564251 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:09:08.564311 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.564362 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.564378 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:09:08.566660 22134 raft_consensus.cc:1261] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Refusing update from remote peer e10ec0d212df4bcf8bdf401861092926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:08.566716 22269 raft_consensus.cc:1261] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Refusing update from remote peer e10ec0d212df4bcf8bdf401861092926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:08.567011 22476 consensus_queue.cc:1035] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:08.567166 22473 consensus_queue.cc:1035] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:08.569718 22481 mvcc.cc:205] Tried to move back new op lower bound from 7332561504525815808 to 7332561504480686080. Current Snapshot: MvccSnapshot[applied={T|T < 7332561504525815808}]
I20260923 15:09:08.570307 22480 mvcc.cc:205] Tried to move back new op lower bound from 7332561504525815808 to 7332561504480686080. Current Snapshot: MvccSnapshot[applied={T|T < 7332561504525815808}]
I20260923 15:09:08.571607 21988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:08.571699 21988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:08.572422 21988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Sending DeleteTablet for 3 replicas of tablet 479d17af68ee4176acd3fe06775cbf15
I20260923 15:09:08.572831 22383 tablet_service.cc:1514] Processing DeleteTablet for tablet 479d17af68ee4176acd3fe06775cbf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:08 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:53034
I20260923 15:09:08.572885 22114 tablet_service.cc:1514] Processing DeleteTablet for tablet 479d17af68ee4176acd3fe06775cbf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:08 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32936
I20260923 15:09:08.572939 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22050
I20260923 15:09:08.573436 22248 tablet_service.cc:1514] Processing DeleteTablet for tablet 479d17af68ee4176acd3fe06775cbf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:08 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:52558
I20260923 15:09:08.573665 22490 tablet_replica.cc:330] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: stopping tablet replica
I20260923 15:09:08.573709 22489 tablet_replica.cc:330] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: stopping tablet replica
I20260923 15:09:08.573814 22490 raft_consensus.cc:2229] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:08.573856 22489 raft_consensus.cc:2229] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:09:08.573990 22490 raft_consensus.cc:2258] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:08.574033 22489 raft_consensus.cc:2258] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:08.574573 22490 ts_tablet_manager.cc:1905] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:08.574647 22489 ts_tablet_manager.cc:1905] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:08.576593 22490 ts_tablet_manager.cc:1918] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:08.576750 22490 log.cc:1196] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/479d17af68ee4176acd3fe06775cbf15
I20260923 15:09:08.576752 22489 ts_tablet_manager.cc:1918] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:08.576819 22489 log.cc:1196] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/479d17af68ee4176acd3fe06775cbf15
I20260923 15:09:08.577149 22489 ts_tablet_manager.cc:1939] T 479d17af68ee4176acd3fe06775cbf15 P e10ec0d212df4bcf8bdf401861092926: Deleting consensus metadata
I20260923 15:09:08.577231 22490 ts_tablet_manager.cc:1939] T 479d17af68ee4176acd3fe06775cbf15 P bdf1565cf61e4f9b9ad8c4de288574af: Deleting consensus metadata
I20260923 15:09:08.577648 21975 catalog_manager.cc:4841] TS e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): tablet 479d17af68ee4176acd3fe06775cbf15 (table test-table [id=35569c596be94b868ab2eefaae9a9ca5]) successfully deleted
I20260923 15:09:08.578033 21974 catalog_manager.cc:4841] TS bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): tablet 479d17af68ee4176acd3fe06775cbf15 (table test-table [id=35569c596be94b868ab2eefaae9a9ca5]) successfully deleted
W20260923 15:09:08.580116 21973 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:43637 (error 108)
I20260923 15:09:08.580296 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22185
I20260923 15:09:08.585240 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22320
I20260923 15:09:08.585381 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.585201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43637 (local address 127.0.0.1:32938)
0923 15:09:08.585273 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.585329 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 15:09:08.585896 21973 catalog_manager.cc:4584] TS 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 479d17af68ee4176acd3fe06775cbf15: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43637: connect: Connection refused (error 111)
I20260923 15:09:08.590691 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 21954
I20260923 15:09:08.596587 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34921
--webserver_interface=127.7.250.126
--webserver_port=37131
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:08.679616 22492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:08.679863 22492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:08.679903 22492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:08.681083 22492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:08.681149 22492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:08.681174 22492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:08.681205 22492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:08.681234 22492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:08.682750 22492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:34921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:08.683034 22492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:08.683264 22492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:08.692163 22498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.692191 22492 server_base.cc:1034] running on GCE node
W20260923 15:09:08.692166 22501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:08.692168 22499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.692806 22492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:08.693184 22492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:08.694394 22492 hybrid_clock.cc:648] HybridClock initialized: now 1790176148694349 us; error 58 us; skew 500 ppm
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:08.697007 22492 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:08.697479 22492 webserver.cc:460] Webserver started at http://127.7.250.126:37131/ using document root <none> and password file <none>
I20260923 15:09:08.697754 22492 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:08.697849 22492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:08.699389 22492 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:08.700346 22508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.700555 22492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:08.700670 22492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "b20f7ba48020487a8a1937341160251f"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "b2151ce069a91fa162094ea12fe625c1"
server_key_iv: "ef7b452d6d102728163dee32f973ed86"
server_key_version: "encryptionkey@0"
I20260923 15:09:08.701113 22492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:08.717980 22492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:08.720182 22492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:08.720347 22492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:08.724677 22492 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:34921
I20260923 15:09:08.724735 22560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:34921 every 8 connection(s)
I20260923 15:09:08.725147 22492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:08.726894 22561 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:09:08.727562 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap starting.
I20260923 15:09:08.731829 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22492
I20260923 15:09:08.732203 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:43637
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:34921
--webserver_port=44853
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:08.736997 22561 log.cc:824] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:08.739020 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:09:08.739336 22561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap complete.
I20260923 15:09:08.741756 22561 raft_consensus.cc:348] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:08.741982 22561 raft_consensus.cc:729] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b20f7ba48020487a8a1937341160251f, State: Initialized, Role: FOLLOWER
I20260923 15:09:08.742144 22561 consensus_queue.cc:260] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:08.742216 22561 raft_consensus.cc:388] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:09:08.742300 22561 raft_consensus.cc:482] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:09:08.742353 22561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:09:08.743440 22561 raft_consensus.cc:504] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:08.743530 22561 leader_election.cc:302] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b20f7ba48020487a8a1937341160251f; no voters:
I20260923 15:09:08.743784 22561 leader_election.cc:288] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:09:08.743831 22565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:09:08.744029 22565 raft_consensus.cc:686] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 LEADER]: Becoming Leader. State: Replica: b20f7ba48020487a8a1937341160251f, State: Running, Role: LEADER
I20260923 15:09:08.744084 22561 sys_catalog.cc:564] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:08.744236 22565 consensus_queue.cc:237] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:08.744625 22566 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b20f7ba48020487a8a1937341160251f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } } }
I20260923 15:09:08.744663 22567 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b20f7ba48020487a8a1937341160251f. Latest consensus state: current_term: 2 leader_uuid: "b20f7ba48020487a8a1937341160251f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } } }
I20260923 15:09:08.744724 22566 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:08.744751 22567 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:08.745044 22572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:08.745896 22572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:09:08.746129 22572 catalog_manager.cc:1259] Loaded cluster ID: 7d199e4a4bab4a15bbc3a55fb573085f
I20260923 15:09:08.746336 22572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:09:08.746610 22572 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 15:09:08.751453 22572 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 15:09:08.754204 22572 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:09:08.754458 22572 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Loaded TSK: 0
I20260923 15:09:08.754700 22572 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:09:08.821573 22563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:08.821818 22563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:08.821872 22563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:08.823153 22563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:08.823220 22563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:08.823267 22563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:09:08.825039 22563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:43637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=44853
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:08.825330 22563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:08.825574 22563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:08.830775 22589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:08.830804 22588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.831071 22563 server_base.cc:1034] running on GCE node
W20260923 15:09:08.830788 22591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.831436 22563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:08.831736 22563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:08.832911 22563 hybrid_clock.cc:648] HybridClock initialized: now 1790176148832899 us; error 35 us; skew 500 ppm
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:08.835578 22563 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:09:08.836021 22563 webserver.cc:460] Webserver started at http://127.7.250.65:44853/ using document root <none> and password file <none>
I20260923 15:09:08.836268 22563 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:08.836357 22563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:08.837801 22563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:08.838582 22598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.838850 22563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:08.838967 22563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "7d6494680a8b4118931c3b450db242ce"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "c0b86714e6c21a19b8eabf17d80b8428"
server_key_iv: "8b66f650850b9ae8b8ab46097dc9a9dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:08.839375 22563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:08.860788 22563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:08.863095 22563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:08.863269 22563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:08.863521 22563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:08.863974 22605 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:09:08.865921 22563 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:09:08.865995 22563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:09:08.866086 22563 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:09:08.866761 22563 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:09:08.866820 22563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:09:08.866884 22605 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Bootstrap starting.
I20260923 15:09:08.874181 22563 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43637
I20260923 15:09:08.874318 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43637 every 8 connection(s)
I20260923 15:09:08.874665 22563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:09:08.877848 22605 log.cc:824] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:08.878191 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22563
I20260923 15:09:08.878528 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:41761
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:34921
--webserver_port=36871
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:08.880630 22605 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:09:08.880988 22605 tablet_bootstrap.cc:492] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Bootstrap complete.
I20260923 15:09:08.882189 22605 ts_tablet_manager.cc:1397] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.014s
I20260923 15:09:08.883961 22605 raft_consensus.cc:348] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
I20260923 15:09:08.884315 22605 raft_consensus.cc:729] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6494680a8b4118931c3b450db242ce, State: Initialized, Role: FOLLOWER
I20260923 15:09:08.884485 22605 consensus_queue.cc:260] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } }
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:08.884773 22605 ts_tablet_manager.cc:1428] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:09:08.889439 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.875465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:36771 (local address 127.7.250.126:34921)
0923 15:09:08.876414 (+ 949us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.876421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.876451 (+ 30us) server_negotiation.cc:407] Connection header received
0923 15:09:08.876563 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.876568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.876607 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.876746 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:08.877385 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.880390 (+ 3005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.881700 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.882335 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.882929 (+ 594us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.885343 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:08.885347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:08.885347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:08.885358 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:08.887137 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.887818 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.887821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.887822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.887945 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.888306 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.888307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.888307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.888588 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:08.888654 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.889155 (+ 501us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.889277 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904,"thread_start_us":81,"threads_started":1}
I20260923 15:09:08.889467 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.875397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:36771)
0923 15:09:08.875781 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.875815 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.876262 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.876785 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.876789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.877078 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:08.877262 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.877266 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.880549 (+ 3283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.880550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.881565 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.881568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.882457 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.882458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.882630 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.883231 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:08.883241 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:08.885219 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:08.887230 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.887239 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.887241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.887671 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.888046 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.888047 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.888047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.888205 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.888680 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:08.888689 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:08.888868 (+ 179us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.888996 (+ 128us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.889200 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":92,"threads_started":1}
I20260923 15:09:08.890452 22714 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34921
I20260923 15:09:08.890576 22714 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:08.890831 22714 heartbeater.cc:511] Master 127.7.250.126:34921 requested a full tablet report, sending...
I20260923 15:09:08.891712 22525 ts_manager.cc:194] Registered new tserver with Master: 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637)
I20260923 15:09:08.893004 22525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:36771
I20260923 15:09:08.894001 22714 heartbeater.cc:503] Master 127.7.250.126:34921 was elected leader, sending a full tablet report...
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:09:08.902166 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.892390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43637 (local address 127.0.0.1:32940)
0923 15:09:08.892651 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:08.892682 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:08.892742 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:08.893025 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:08.893032 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:08.893470 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:08.893543 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.893546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.894675 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.894677 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.895664 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:08.895669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.896205 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.896206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:08.896296 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.896807 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:08.896813 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:08.898715 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:08.900819 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.900829 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.900831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.901067 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.901327 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:08.901328 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:08.901329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:08.901402 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:08.901735 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:08.901744 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:08.901804 (+ 60us) client_negotiation.cc:769] Sending connection context
0923 15:09:08.901911 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 15:09:08.902058 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":135,"threads_started":1}
I20260923 15:09:08.902166 22722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:08.892463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32940 (local address 127.7.250.65:43637)
0923 15:09:08.892863 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:08.892867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:08.892877 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:09:08.892911 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:08.892913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:08.892930 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:08.892999 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:08.893627 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.894597 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.895773 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:08.896122 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:08.896184 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:08.898834 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:08.898850 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:08.898851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:08.898859 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:08.900673 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.901173 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.901177 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.901177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.901253 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:08.901476 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:08.901477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:08.901477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:08.901659 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:08.901761 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:08.901950 (+ 189us) server_negotiation.cc:299] Negotiation successful
0923 15:09:08.902065 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":96,"threads_started":1}
I20260923 15:09:08.902809 22647 tablet_service.cc:1514] Processing DeleteTablet for tablet 479d17af68ee4176acd3fe06775cbf15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:08 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32940
I20260923 15:09:08.903376 22723 tablet_replica.cc:330] stopping tablet replica
I20260923 15:09:08.903501 22723 raft_consensus.cc:2229] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:08.903594 22723 raft_consensus.cc:2258] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:08.903945 22723 ts_tablet_manager.cc:1905] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:08.906307 22723 ts_tablet_manager.cc:1918] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:08.906392 22723 log.cc:1196] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/479d17af68ee4176acd3fe06775cbf15
I20260923 15:09:08.906633 22723 ts_tablet_manager.cc:1939] T 479d17af68ee4176acd3fe06775cbf15 P 7d6494680a8b4118931c3b450db242ce: Deleting consensus metadata
I20260923 15:09:08.907296 22510 catalog_manager.cc:4841] TS 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637): tablet 479d17af68ee4176acd3fe06775cbf15 (table test-table [id=35569c596be94b868ab2eefaae9a9ca5]) successfully deleted
W20260923 15:09:08.968818 22719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:08.969040 22719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:08.969066 22719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:08.970302 22719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:08.970359 22719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:08.970387 22719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:09:08.971894 22719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:41761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=36871
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:08.972155 22719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:08.972398 22719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:08.977633 22729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.977661 22719 server_base.cc:1034] running on GCE node
W20260923 15:09:08.977628 22728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:08.977633 22731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:08.978080 22719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:08.978411 22719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:08.979586 22719 hybrid_clock.cc:648] HybridClock initialized: now 1790176148979574 us; error 29 us; skew 500 ppm
Sep 23 15:09:08 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:08.982337 22719 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:09:08.982837 22719 webserver.cc:460] Webserver started at http://127.7.250.66:36871/ using document root <none> and password file <none>
I20260923 15:09:08.983109 22719 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:08.983196 22719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:08.984654 22719 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:08.985267 22738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.985428 22719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:09:08.985558 22719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "8f904add515d0f637066cf03f4afc9e2"
server_key_iv: "a09877fce13f9bde325ebc80565ac07a"
server_key_version: "encryptionkey@0"
I20260923 15:09:08.985911 22719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:08.994971 22719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:08.997001 22719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:08.997172 22719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:08.997406 22719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:08.997761 22719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:08.997818 22719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:08.997852 22719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:08.997879 22719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.004365 22719 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:41761
I20260923 15:09:09.004386 22851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:41761 every 8 connection(s)
I20260923 15:09:09.004951 22719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:09:09.005175 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22719
I20260923 15:09:09.005599 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:38253
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:34921
--webserver_port=39395
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176148, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:09.017309 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.005778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:55281 (local address 127.7.250.126:34921)
0923 15:09:09.005828 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.005831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.006163 (+ 332us) server_negotiation.cc:407] Connection header received
0923 15:09:09.006754 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.006756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.006775 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.006871 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:09.008337 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.008942 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.010713 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.011018 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.011096 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.014068 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:09.014070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:09.014071 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:09.014079 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:09.015750 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:09.016228 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:09.016232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:09.016232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:09.016296 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:09.016575 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:09.016576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:09.016576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:09.016706 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:09.016779 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.017091 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.017197 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:09:09.017508 22854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.005809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:55281)
0923 15:09:09.006006 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.006053 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.006643 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.006955 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:09.006960 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:09.007857 (+ 897us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:09.008192 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.008199 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.009079 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.009080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.010585 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.010589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.011138 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.011140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.011333 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.011978 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:09.011989 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:09.013946 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:09.015857 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:09.015867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:09.015870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:09.016136 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:09.016371 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:09.016372 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:09.016373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:09.016498 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:09.016783 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:09.016790 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:09.016950 (+ 160us) client_negotiation.cc:769] Sending connection context
0923 15:09:09.017058 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 15:09:09.017235 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":84,"threads_started":1}
I20260923 15:09:09.018100 22852 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34921
I20260923 15:09:09.018217 22852 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:09.018460 22852 heartbeater.cc:511] Master 127.7.250.126:34921 requested a full tablet report, sending...
I20260923 15:09:09.018923 22525 ts_manager.cc:194] Registered new tserver with Master: bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761)
I20260923 15:09:09.019335 22525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:55281
W20260923 15:09:09.093676 22855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:09.094189 22855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:09.094244 22855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:09.096380 22855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:09.096467 22855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:09.096520 22855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:09:09.098977 22855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:38253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=39395
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:09.099316 22855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:09.099641 22855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:09.105777 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.105777 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.105788 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:09.106593 22855 server_base.cc:1034] running on GCE node
I20260923 15:09:09.106827 22855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:09.107172 22855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:09.108508 22855 hybrid_clock.cc:648] HybridClock initialized: now 1790176149108492 us; error 31 us; skew 500 ppm
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:09.111992 22855 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:09:09.112699 22855 webserver.cc:460] Webserver started at http://127.7.250.67:39395/ using document root <none> and password file <none>
I20260923 15:09:09.112986 22855 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:09.113081 22855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:09.114666 22855 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:09.115464 22870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.115747 22855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:09.115859 22855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "e10ec0d212df4bcf8bdf401861092926"
format_stamp: "Formatted at 2026-09-23 15:09:08 on dist-test-slave-bb3x"
server_key: "590de1a6454fb071c5bfc4f5c40b3690"
server_key_iv: "607f575a795d2e03bda887b4e894b0dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:09.116264 22855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:09.130225 22855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:09.134099 22855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:09.134470 22855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:09.134902 22855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:09.135367 22855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:09.135442 22855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.135488 22855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:09.135517 22855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.142453 22855 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:38253
I20260923 15:09:09.142554 22984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:38253 every 8 connection(s)
I20260923 15:09:09.142935 22855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:09.152556 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 22855
I20260923 15:09:09.155418 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.143941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:60777 (local address 127.7.250.126:34921)
0923 15:09:09.144015 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.144018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.144491 (+ 473us) server_negotiation.cc:407] Connection header received
0923 15:09:09.145316 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.145317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.145329 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.145389 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:09.147158 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.147699 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.148840 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.149127 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.149197 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.151570 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:09.151571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:09.151572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:09.151580 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:09.153753 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:09.154308 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:09.154310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:09.154310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:09.154390 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:09.154696 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:09.154697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:09.154698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:09.154827 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:09.154889 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.155263 (+ 374us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.155343 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:09:09.155772 22987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.144061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:60777)
0923 15:09:09.144294 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.144340 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.145203 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.145457 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:09.145461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:09.146463 (+ 1002us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:09.146990 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.146999 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.147842 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.147843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.148725 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.148729 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.149222 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.149223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.149342 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.149861 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:09.149870 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:09.151440 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:09.153869 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:09.153881 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:09.153883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:09.154191 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:09.154468 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:09.154468 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:09.154469 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:09.154620 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:09.154900 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:09.154907 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:09.155090 (+ 183us) client_negotiation.cc:769] Sending connection context
0923 15:09:09.155212 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 15:09:09.155406 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":140,"threads_started":1}
I20260923 15:09:09.156756 22985 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34921
I20260923 15:09:09.156885 22985 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:09.157140 22985 heartbeater.cc:511] Master 127.7.250.126:34921 requested a full tablet report, sending...
I20260923 15:09:09.157609 22525 ts_manager.cc:194] Registered new tserver with Master: e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
I20260923 15:09:09.158092 22525 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:60777
I20260923 15:09:09.161788 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.153086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.7.250.126:34921)
0923 15:09:09.153486 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.153488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.153495 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:09:09.153517 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.153518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.153529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.153586 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:09.154636 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.155387 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.156703 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.157006 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.157082 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.158750 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:09.158752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:09.158753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:09.158761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:09.160322 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:09.160770 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:09.160772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:09.160772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:09.160869 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:09.161169 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:09.161170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:09.161170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:09.161306 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:09.161513 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.161578 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.161672 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":104,"threads_started":1}
I20260923 15:09:09.162880 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:09:09.166442 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.163258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38482 (local address 127.7.250.126:34921)
0923 15:09:09.163347 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.163349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.163546 (+ 197us) server_negotiation.cc:407] Connection header received
0923 15:09:09.163666 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.163666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.163717 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.163816 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:09.163925 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.164482 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.165501 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.165728 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.165815 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.166009 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:09.166127 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:09:09.166229 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.166247 (+ 18us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.166310 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 15:09:09.166746 22525 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38482:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:09:09.169405 22525 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:09:09.172984 22647 tablet_service.cc:1467] Processing CreateTablet for tablet 363686ebda1541e589d45d403d61ad97 (DEFAULT_TABLE table=test-table [id=eb4d7c4bc10a4e89aa05bda40ef6dd6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:09.173296 22647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363686ebda1541e589d45d403d61ad97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:09.175298 22605 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Bootstrap starting.
I20260923 15:09:09.175647 22605 tablet_bootstrap.cc:654] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:09.176257 22605 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: No bootstrap required, opened a new log
I20260923 15:09:09.176311 22605 ts_tablet_manager.cc:1397] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:09.176699 22605 raft_consensus.cc:348] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.176782 22605 raft_consensus.cc:374] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:09.176834 22605 raft_consensus.cc:729] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6494680a8b4118931c3b450db242ce, State: Initialized, Role: FOLLOWER
I20260923 15:09:09.176892 22605 consensus_queue.cc:260] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.176992 22605 ts_tablet_manager.cc:1428] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 15:09:09.177527 22721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.172334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:38253 (local address 127.0.0.1:53050)
0923 15:09:09.172455 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.172468 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.172514 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.173026 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:09.173027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:09.173029 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:09.173135 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.173138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.174764 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.174765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.176159 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.176163 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.177173 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.177174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.177262 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.177299 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 15:09:09.177333 (+ 34us) client_negotiation.cc:240] Negotiation successful
0923 15:09:09.177396 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260923 15:09:09.177675 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.172359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41761 (local address 127.0.0.1:52568)
0923 15:09:09.172557 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.172567 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.172608 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.173635 (+ 1027us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:09.173637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:09.173639 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:09.173695 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.173723 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.175049 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.175050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.176261 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.176263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.177301 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.177302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.177395 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.177400 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:09:09.177499 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 15:09:09.177592 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":130,"threads_started":1}
I20260923 15:09:09.177954 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.172406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53050 (local address 127.7.250.67:38253)
0923 15:09:09.172809 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.172815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.172826 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:09:09.172873 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.172875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.172892 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.172980 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:09.173238 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.174620 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.176257 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.177067 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.177644 (+ 577us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.177722 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.177753 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.177884 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":110,"threads_started":1}
I20260923 15:09:09.177959 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.172459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52568 (local address 127.7.250.66:41761)
0923 15:09:09.173314 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.173320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.173331 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:09:09.173401 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.173403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.173419 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.173502 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:09.173811 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.174900 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.176364 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.177040 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.177133 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.177243 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.177815 (+ 572us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.177879 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":89,"threads_started":1}
I20260923 15:09:09.178669 22786 tablet_service.cc:1467] Processing CreateTablet for tablet 363686ebda1541e589d45d403d61ad97 (DEFAULT_TABLE table=test-table [id=eb4d7c4bc10a4e89aa05bda40ef6dd6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:09.178678 22919 tablet_service.cc:1467] Processing CreateTablet for tablet 363686ebda1541e589d45d403d61ad97 (DEFAULT_TABLE table=test-table [id=eb4d7c4bc10a4e89aa05bda40ef6dd6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:09.179029 22786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363686ebda1541e589d45d403d61ad97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:09.179040 22919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363686ebda1541e589d45d403d61ad97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:09.182693 22995 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Bootstrap starting.
I20260923 15:09:09.183439 22995 tablet_bootstrap.cc:654] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:09.183969 22995 log.cc:824] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:09.184203 22996 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Bootstrap starting.
I20260923 15:09:09.184821 22996 tablet_bootstrap.cc:654] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:09.184828 22995 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: No bootstrap required, opened a new log
I20260923 15:09:09.184984 22995 ts_tablet_manager.cc:1397] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:09:09.185364 22996 log.cc:824] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:09.186738 22996 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: No bootstrap required, opened a new log
I20260923 15:09:09.186852 22996 ts_tablet_manager.cc:1397] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260923 15:09:09.187251 22995 raft_consensus.cc:348] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.187408 22995 raft_consensus.cc:374] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:09.187433 22995 raft_consensus.cc:729] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10ec0d212df4bcf8bdf401861092926, State: Initialized, Role: FOLLOWER
I20260923 15:09:09.187600 22995 consensus_queue.cc:260] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.187937 22995 ts_tablet_manager.cc:1428] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:09.187961 22985 heartbeater.cc:503] Master 127.7.250.126:34921 was elected leader, sending a full tablet report...
I20260923 15:09:09.188774 22996 raft_consensus.cc:348] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.188992 22996 raft_consensus.cc:374] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:09.189037 22996 raft_consensus.cc:729] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdf1565cf61e4f9b9ad8c4de288574af, State: Initialized, Role: FOLLOWER
I20260923 15:09:09.189162 22996 consensus_queue.cc:260] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.189508 22996 ts_tablet_manager.cc:1428] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:09.189533 22852 heartbeater.cc:503] Master 127.7.250.126:34921 was elected leader, sending a full tablet report...
W20260923 15:09:09.255664 22853 tablet.cc:2369] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:09:09.256664 22720 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:09.256855 22720 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.257388 22720 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
I20260923 15:09:09.260654 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.257656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:38253 (local address 127.7.250.65:46697)
0923 15:09:09.257741 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.257755 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.257825 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.258054 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:09.258056 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:09.258058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:09.258146 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.258148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.258912 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.258913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.259739 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.259742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.260340 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.260341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.260426 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.260449 (+ 23us) client_negotiation.cc:769] Sending connection context
0923 15:09:09.260519 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 15:09:09.260571 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":1}
I20260923 15:09:09.260654 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.257735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:46697 (local address 127.7.250.67:38253)
0923 15:09:09.257786 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.257788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.257800 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:09:09.257917 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.257917 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.257926 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.258016 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:09.258237 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.258814 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.259837 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.260242 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.260350 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.260361 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.260524 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.260577 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:09:09.260803 23001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.257670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41761 (local address 127.7.250.65:44955)
0923 15:09:09.257897 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.257908 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.257988 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.258153 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:09.258153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:09.258155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:09.258235 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.258237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.258934 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.258935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.259796 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:09.259798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.260496 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.260497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:09.260581 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.260585 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:09.260667 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 15:09:09.260709 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":2,"thread_start_us":137,"threads_started":1}
I20260923 15:09:09.260803 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.257725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:44955 (local address 127.7.250.66:41761)
0923 15:09:09.257770 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.257772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.257979 (+ 207us) server_negotiation.cc:407] Connection header received
0923 15:09:09.258067 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.258069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.258085 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.258150 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:09.258339 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.258822 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.259899 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.260345 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.260491 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.260502 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.260675 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.260715 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:09:09.261077 22939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363686ebda1541e589d45d403d61ad97" candidate_uuid: "7d6494680a8b4118931c3b450db242ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10ec0d212df4bcf8bdf401861092926" is_pre_election: true
I20260923 15:09:09.261277 22939 raft_consensus.cc:2454] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6494680a8b4118931c3b450db242ce in term 0.
I20260923 15:09:09.261364 22806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363686ebda1541e589d45d403d61ad97" candidate_uuid: "7d6494680a8b4118931c3b450db242ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" is_pre_election: true
I20260923 15:09:09.261544 22602 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce, e10ec0d212df4bcf8bdf401861092926; no voters:
I20260923 15:09:09.261600 22806 raft_consensus.cc:2454] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d6494680a8b4118931c3b450db242ce in term 0.
I20260923 15:09:09.261708 22720 raft_consensus.cc:2781] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:09:09.261760 22720 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:09:09.261776 22720 raft_consensus.cc:3037] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:09.262728 22720 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.262888 22720 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 1 election: Requested vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
I20260923 15:09:09.263187 22806 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363686ebda1541e589d45d403d61ad97" candidate_uuid: "7d6494680a8b4118931c3b450db242ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
I20260923 15:09:09.263190 22939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363686ebda1541e589d45d403d61ad97" candidate_uuid: "7d6494680a8b4118931c3b450db242ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e10ec0d212df4bcf8bdf401861092926"
I20260923 15:09:09.263309 22939 raft_consensus.cc:3037] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:09.263321 22806 raft_consensus.cc:3037] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:09.264200 22939 raft_consensus.cc:2454] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6494680a8b4118931c3b450db242ce in term 1.
I20260923 15:09:09.264395 22806 raft_consensus.cc:2454] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d6494680a8b4118931c3b450db242ce in term 1.
I20260923 15:09:09.264601 22601 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce, bdf1565cf61e4f9b9ad8c4de288574af; no voters:
I20260923 15:09:09.264715 22720 raft_consensus.cc:2781] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:09.264907 22720 raft_consensus.cc:686] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 LEADER]: Becoming Leader. State: Replica: 7d6494680a8b4118931c3b450db242ce, State: Running, Role: LEADER
I20260923 15:09:09.265003 22720 consensus_queue.cc:237] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.266036 22525 catalog_manager.cc:5495] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d6494680a8b4118931c3b450db242ce (127.7.250.65). New cstate: current_term: 1 leader_uuid: "7d6494680a8b4118931c3b450db242ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } health_report { overall_health: HEALTHY } } }
I20260923 15:09:09.276511 22722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.273360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.7.250.65:43637)
0923 15:09:09.273560 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:09.273562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:09.273572 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:09:09.273624 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:09.273625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:09.273646 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:09.273711 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:09.274033 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.274706 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.275521 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:09.275738 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:09.275807 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:09.276049 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:09.276170 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:09:09.276261 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:09.276407 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 15:09:09.276438 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173}
I20260923 15:09:09.278721 22806 raft_consensus.cc:1261] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Refusing update from remote peer 7d6494680a8b4118931c3b450db242ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:09.278777 22939 raft_consensus.cc:1261] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Refusing update from remote peer 7d6494680a8b4118931c3b450db242ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:09.279230 22720 consensus_queue.cc:1035] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:09.279419 23002 consensus_queue.cc:1035] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:09.282939 23006 mvcc.cc:205] Tried to move back new op lower bound from 7332561507442094080 to 7332561507390402560. Current Snapshot: MvccSnapshot[applied={T|T < 7332561507442094080}]
I20260923 15:09:09.283077 23007 mvcc.cc:205] Tried to move back new op lower bound from 7332561507442094080 to 7332561507390402560. Current Snapshot: MvccSnapshot[applied={T|T < 7332561507442094080}]
I20260923 15:09:09.285861 22525 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:09.285974 22525 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38482:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:09.286933 22525 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Sending DeleteTablet for 3 replicas of tablet 363686ebda1541e589d45d403d61ad97
I20260923 15:09:09.287431 22786 tablet_service.cc:1514] Processing DeleteTablet for tablet 363686ebda1541e589d45d403d61ad97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:09 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:52568
I20260923 15:09:09.287705 23015 tablet_replica.cc:330] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: stopping tablet replica
I20260923 15:09:09.287678 22647 tablet_service.cc:1514] Processing DeleteTablet for tablet 363686ebda1541e589d45d403d61ad97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:09 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:32940
I20260923 15:09:09.287784 22723 tablet_replica.cc:330] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: stopping tablet replica
I20260923 15:09:09.287846 22723 raft_consensus.cc:2229] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:09:09.287837 23015 raft_consensus.cc:2229] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:09.287981 22723 raft_consensus.cc:2258] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:09.288090 23015 raft_consensus.cc:2258] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:09.288444 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22563
I20260923 15:09:09.288506 22919 tablet_service.cc:1514] Processing DeleteTablet for tablet 363686ebda1541e589d45d403d61ad97 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:09 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:53050
I20260923 15:09:09.288676 22723 ts_tablet_manager.cc:1905] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:09.289280 23016 tablet_replica.cc:330] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: stopping tablet replica
I20260923 15:09:09.289410 23016 raft_consensus.cc:2229] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:09.289587 23016 raft_consensus.cc:2258] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:09.290081 23016 ts_tablet_manager.cc:1905] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:09.290180 23015 ts_tablet_manager.cc:1905] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:09.293687 23015 ts_tablet_manager.cc:1918] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:09.293687 23016 ts_tablet_manager.cc:1918] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:09.293810 23015 log.cc:1196] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/363686ebda1541e589d45d403d61ad97
I20260923 15:09:09.293823 23016 log.cc:1196] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/363686ebda1541e589d45d403d61ad97
I20260923 15:09:09.294173 23015 ts_tablet_manager.cc:1939] T 363686ebda1541e589d45d403d61ad97 P bdf1565cf61e4f9b9ad8c4de288574af: Deleting consensus metadata
I20260923 15:09:09.294173 23016 ts_tablet_manager.cc:1939] T 363686ebda1541e589d45d403d61ad97 P e10ec0d212df4bcf8bdf401861092926: Deleting consensus metadata
I20260923 15:09:09.294925 22512 catalog_manager.cc:4841] TS e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): tablet 363686ebda1541e589d45d403d61ad97 (table test-table [id=eb4d7c4bc10a4e89aa05bda40ef6dd6a]) successfully deleted
I20260923 15:09:09.294942 22511 catalog_manager.cc:4841] TS bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): tablet 363686ebda1541e589d45d403d61ad97 (table test-table [id=eb4d7c4bc10a4e89aa05bda40ef6dd6a]) successfully deleted
W20260923 15:09:09.295392 22510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:43637 (error 108)
I20260923 15:09:09.295984 22991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.295826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43637 (local address 127.0.0.1:32954)
0923 15:09:09.295893 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.295938 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:09:09.296190 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22719
W20260923 15:09:09.296182 22510 catalog_manager.cc:4584] TS 7d6494680a8b4118931c3b450db242ce (127.7.250.65:43637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 363686ebda1541e589d45d403d61ad97: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43637: connect: Connection refused (error 111)
I20260923 15:09:09.301874 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22855
I20260923 15:09:09.307458 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 22492
I20260923 15:09:09.313323 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34921
--webserver_interface=127.7.250.126
--webserver_port=37131
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:34921
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:09.398166 23018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:09.398439 23018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:09.398476 23018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:09.399658 23018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:09.399731 23018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:09.399758 23018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:09.399789 23018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:09.399812 23018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:09.401316 23018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:34921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=37131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:09.401592 23018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:09.401818 23018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:09.407385 23023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.407418 23026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.407394 23024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:09.407732 23018 server_base.cc:1034] running on GCE node
I20260923 15:09:09.408087 23018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:09.408491 23018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:09.409706 23018 hybrid_clock.cc:648] HybridClock initialized: now 1790176149409693 us; error 27 us; skew 500 ppm
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:09.412501 23018 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:09.413079 23018 webserver.cc:460] Webserver started at http://127.7.250.126:37131/ using document root <none> and password file <none>
I20260923 15:09:09.413398 23018 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:09.413489 23018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:09.415169 23018 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:09.416025 23033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.416262 23018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:09.416392 23018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "b20f7ba48020487a8a1937341160251f"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "b2151ce069a91fa162094ea12fe625c1"
server_key_iv: "ef7b452d6d102728163dee32f973ed86"
server_key_version: "encryptionkey@0"
I20260923 15:09:09.416801 23018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:09.428457 23018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:09.430651 23018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:09.430821 23018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:09.435382 23018 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:34921
I20260923 15:09:09.435427 23085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:34921 every 8 connection(s)
I20260923 15:09:09.435844 23018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:09.437419 23086 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:09:09.438040 23086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap starting.
I20260923 15:09:09.439143 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23018
I20260923 15:09:09.439455 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:43637
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:34921
--webserver_port=44853
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:09.447036 23086 log.cc:824] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:09.449227 23086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:09:09.449548 23086 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f: Bootstrap complete.
I20260923 15:09:09.451833 23086 raft_consensus.cc:348] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:09.452056 23086 raft_consensus.cc:729] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b20f7ba48020487a8a1937341160251f, State: Initialized, Role: FOLLOWER
I20260923 15:09:09.452231 23086 consensus_queue.cc:260] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:09.452291 23086 raft_consensus.cc:388] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:09:09.452354 23086 raft_consensus.cc:482] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:09:09.452395 23086 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 2 FOLLOWER]: Advancing to term 3
I20260923 15:09:09.453384 23086 raft_consensus.cc:504] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:09.453469 23086 leader_election.cc:302] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b20f7ba48020487a8a1937341160251f; no voters:
I20260923 15:09:09.453658 23086 leader_election.cc:288] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 15:09:09.453696 23090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 3 FOLLOWER]: Leader election won for term 3
I20260923 15:09:09.453859 23090 raft_consensus.cc:686] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [term 3 LEADER]: Becoming Leader. State: Replica: b20f7ba48020487a8a1937341160251f, State: Running, Role: LEADER
I20260923 15:09:09.453924 23086 sys_catalog.cc:564] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:09.454015 23090 consensus_queue.cc:237] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } }
I20260923 15:09:09.454425 23092 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b20f7ba48020487a8a1937341160251f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } } }
I20260923 15:09:09.454427 23090 sys_catalog.cc:455] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b20f7ba48020487a8a1937341160251f. Latest consensus state: current_term: 3 leader_uuid: "b20f7ba48020487a8a1937341160251f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b20f7ba48020487a8a1937341160251f" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34921 } } }
I20260923 15:09:09.454615 23092 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:09.454625 23090 sys_catalog.cc:458] T 00000000000000000000000000000000 P b20f7ba48020487a8a1937341160251f [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:09.454955 23099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:09.455749 23099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:09:09.455920 23099 catalog_manager.cc:1259] Loaded cluster ID: 7d199e4a4bab4a15bbc3a55fb573085f
I20260923 15:09:09.456135 23099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 15:09:09.456343 23099 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4b5b11d9ad google::LogMessage::Fail() at ??:0
@ 0x7f4b5b12088a google::LogMessage::SendToLog() at ??:0
@ 0x7f4b5b11d4de google::LogMessage::Flush() at ??:0
@ 0x7f4b5b11ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4b6303e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4b630478b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4b5cfeb0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4b5cfe0954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4b5de446db start_thread at ??:0
@ 0x7f4b59c1b61f clone at ??:0
W20260923 15:09:09.533059 23088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:09.533291 23088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:09.533330 23088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:09.534662 23088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:09.534727 23088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:09.534770 23088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:09:09.536273 23088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:43637
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=44853
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:09.536558 23088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:09.536798 23088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:09.543260 23123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:09.543300 23088 server_base.cc:1034] running on GCE node
W20260923 15:09:09.543277 23125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.543452 23122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:09.543752 23088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:09.544027 23088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:09.545193 23088 hybrid_clock.cc:648] HybridClock initialized: now 1790176149545176 us; error 33 us; skew 500 ppm
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:09.547984 23088 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:09:09.548522 23088 webserver.cc:460] Webserver started at http://127.7.250.65:44853/ using document root <none> and password file <none>
I20260923 15:09:09.548792 23088 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:09.548866 23088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:09.550590 23088 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:09.551295 23132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.551477 23088 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:09.551584 23088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "7d6494680a8b4118931c3b450db242ce"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "c0b86714e6c21a19b8eabf17d80b8428"
server_key_iv: "8b66f650850b9ae8b8ab46097dc9a9dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:09.551939 23088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:09.585114 23088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:09.587873 23088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:09.588037 23088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:09.588310 23088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:09.588891 23139 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:09:09.590623 23088 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:09:09.590682 23088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:09:09.590718 23088 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:09:09.591657 23088 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:09:09.591704 23088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:09:09.591816 23139 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Bootstrap starting.
I20260923 15:09:09.593571 23139 log.cc:824] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:09.596731 23139 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:09:09.597029 23139 tablet_bootstrap.cc:492] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Bootstrap complete.
I20260923 15:09:09.597218 23139 ts_tablet_manager.cc:1397] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260923 15:09:09.598733 23088 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43637
I20260923 15:09:09.598845 23247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43637 every 8 connection(s)
I20260923 15:09:09.599117 23088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:09:09.599185 23139 raft_consensus.cc:348] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.599571 23139 raft_consensus.cc:729] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6494680a8b4118931c3b450db242ce, State: Initialized, Role: FOLLOWER
I20260923 15:09:09.599730 23139 consensus_queue.cc:260] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:09.599978 23139 ts_tablet_manager.cc:1428] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce: Time spent starting tablet: real 0.003s user 0.002s sys 0.003s
I20260923 15:09:09.605198 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23088
I20260923 15:09:09.605487 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:41761
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:34921
--webserver_port=36871
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:09.658885 23250 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:09:09.599766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:35519)
0923 15:09:09.599955 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.600000 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:09.600153 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:09.658642 (+ 58489us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
W20260923 15:09:09.659201 23248 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:34921 (0 consecutive failures): Network error: Failed to ping master at 127.7.250.126:34921: Client connection negotiation failed: client connection to 127.7.250.126:34921: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:09:09.659655 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.659521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:54165)
0923 15:09:09.659586 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.659633 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:09:09.659979 23250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.659899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:48397)
0923 15:09:09.659941 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.659964 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:09:09.660107 23248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:09:09.692121 23252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:09.692371 23252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:09.692399 23252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:09.693717 23252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:09.693763 23252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:09.693794 23252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:09:09.695520 23252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:41761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=36871
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:09.695924 23252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:09.696187 23252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:09.701642 23258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.701642 23260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.701668 23257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:09.702103 23252 server_base.cc:1034] running on GCE node
I20260923 15:09:09.702334 23252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:09.702538 23252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:09.703696 23252 hybrid_clock.cc:648] HybridClock initialized: now 1790176149703683 us; error 29 us; skew 500 ppm
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:09.706162 23252 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:09:09.706636 23252 webserver.cc:460] Webserver started at http://127.7.250.66:36871/ using document root <none> and password file <none>
I20260923 15:09:09.706847 23252 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:09.706888 23252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:09.708493 23252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:09:09.709090 23267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.709306 23252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:09.709390 23252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "bdf1565cf61e4f9b9ad8c4de288574af"
format_stamp: "Formatted at 2026-09-23 15:09:07 on dist-test-slave-bb3x"
server_key: "8f904add515d0f637066cf03f4afc9e2"
server_key_iv: "a09877fce13f9bde325ebc80565ac07a"
server_key_version: "encryptionkey@0"
I20260923 15:09:09.709757 23252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:09.724639 23252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:09.727361 23252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:09.727519 23252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:09.727811 23252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:09.728164 23252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:09.728226 23252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.728267 23252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:09.728299 23252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.734675 23252 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:41761
I20260923 15:09:09.734794 23380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:41761 every 8 connection(s)
I20260923 15:09:09.735157 23252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:09:09.736364 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.735961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:34281)
0923 15:09:09.736147 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.736220 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":117,"threads_started":1}
W20260923 15:09:09.736888 23381 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:34921 (0 consecutive failures): Network error: Failed to ping master at 127.7.250.126:34921: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
I20260923 15:09:09.737239 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.737100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:55997)
0923 15:09:09.737180 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.737215 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 15:09:09.737702 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.737575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:43947)
0923 15:09:09.737648 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.737679 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260923 15:09:09.737856 23381 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:09:09.740860 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23252
I20260923 15:09:09.741097 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:38253
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:34921
--webserver_port=39395
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:09.824668 23384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:09.824891 23384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:09.824918 23384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:09.826098 23384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:09.826145 23384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:09.826174 23384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:09:09.827744 23384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:38253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=39395
--tserver_master_addrs=127.7.250.126:34921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:09.828011 23384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:09.828246 23384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:09.833379 23390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.833421 23392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:09.833420 23389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:09.833748 23384 server_base.cc:1034] running on GCE node
I20260923 15:09:09.834015 23384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:09.834301 23384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:09.835484 23384 hybrid_clock.cc:648] HybridClock initialized: now 1790176149835471 us; error 29 us; skew 500 ppm
Sep 23 15:09:09 dist-test-slave-bb3x krb5kdc[21941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:09.838075 23384 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:09:09.838590 23384 webserver.cc:460] Webserver started at http://127.7.250.67:39395/ using document root <none> and password file <none>
I20260923 15:09:09.838838 23384 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:09.838910 23384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:09.840317 23384 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:09.840962 23399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.841315 23384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:09.841449 23384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "e10ec0d212df4bcf8bdf401861092926"
format_stamp: "Formatted at 2026-09-23 15:09:08 on dist-test-slave-bb3x"
server_key: "590de1a6454fb071c5bfc4f5c40b3690"
server_key_iv: "607f575a795d2e03bda887b4e894b0dd"
server_key_version: "encryptionkey@0"
I20260923 15:09:09.841840 23384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:09.860109 23384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:09.862417 23384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:09.862588 23384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:09.862844 23384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:09.863171 23384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:09.863227 23384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.863269 23384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:09.863314 23384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:09.869486 23384 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:38253
I20260923 15:09:09.869545 23512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:38253 every 8 connection(s)
I20260923 15:09:09.869933 23384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:09:09.871145 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.870656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:52953)
0923 15:09:09.870916 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.870997 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":89,"threads_started":1}
W20260923 15:09:09.871598 23513 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:34921 (0 consecutive failures): Network error: Failed to ping master at 127.7.250.126:34921: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
I20260923 15:09:09.871937 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.871802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:58095)
0923 15:09:09.871889 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.871919 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260923 15:09:09.872244 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:09.872162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:37525)
0923 15:09:09.872184 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:09.872202 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260923 15:09:09.872387 23513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:09:09.876202 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23384
I20260923 15:09:10.660799 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.660541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:49017)
0923 15:09:10.660694 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:10.660738 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260923 15:09:10.735984 23518 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:10.736109 23518 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:10.736455 23518 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
I20260923 15:09:10.738413 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.738153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:42947)
0923 15:09:10.738298 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:10.738346 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
Sep 23 15:09:10 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.66@KRBTEST.COM
Sep 23 15:09:10 dist-test-slave-bb3x krb5kdc[21941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176149, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.67@KRBTEST.COM
I20260923 15:09:10.747177 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.736717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:40151 (local address 127.7.250.66:41761)
0923 15:09:10.736843 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:10.736855 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:10.736878 (+ 23us) server_negotiation.cc:407] Connection header received
0923 15:09:10.737319 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:10.737322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:10.737363 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:10.737472 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:10.738438 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.739341 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.740300 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.740702 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.740823 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:10.743172 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:10.743177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:10.743177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:10.743190 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:10.745170 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:10.745634 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:10.745637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:10.745637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:10.745755 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:10.746086 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:10.746087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:10.746087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:10.746448 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:10.746537 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:10.746964 (+ 427us) server_negotiation.cc:299] Negotiation successful
0923 15:09:10.747081 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":45,"threads_started":1}
I20260923 15:09:10.747171 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.736859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:40319 (local address 127.7.250.67:38253)
0923 15:09:10.736974 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:10.736982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:10.736998 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:09:10.737394 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:10.737396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:10.737474 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:10.737608 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:10.738793 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.739769 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.740518 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.740930 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.741044 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:10.743491 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:10.743494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:10.743494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:10.743538 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:10.745419 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:10.745768 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:10.745770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:10.745771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:10.745883 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:10.746085 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:10.746086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:10.746087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:10.746448 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:10.746537 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:10.746907 (+ 370us) server_negotiation.cc:299] Negotiation successful
0923 15:09:10.747057 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":45,"threads_started":1}
I20260923 15:09:10.747216 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.736647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:38253 (local address 127.7.250.65:40319)
0923 15:09:10.736785 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:10.736795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:10.736853 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:10.737616 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:10.737617 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:10.738596 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:10.738663 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:10.738665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.739873 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.739874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:10.740428 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:10.740430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.741014 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.741016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:10.741083 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:10.741637 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:10.741642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:10.743385 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:10.745500 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:10.745501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:10.745501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:10.745690 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:10.745947 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:10.745948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:10.745948 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:10.746018 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:10.746527 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:10.746530 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:10.746708 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 15:09:10.746827 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 15:09:10.747106 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":163,"thread_start_us":67,"threads_started":1}
I20260923 15:09:10.747205 23517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.736635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41761 (local address 127.7.250.65:40151)
0923 15:09:10.736689 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:10.736701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:10.736762 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:10.737545 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:10.737548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:10.737986 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:10.738312 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:10.738319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.739448 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.739449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:10.740195 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:10.740197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:10.740797 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:10.740798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:10.740930 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:10.741486 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:10.741496 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:10.743048 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:10.745263 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:10.745270 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:10.745273 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:10.745551 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:10.745874 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:10.745875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:10.745875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:10.746018 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:10.746524 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:10.746530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:10.746708 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 15:09:10.746827 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 15:09:10.747052 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":23}
W20260923 15:09:10.748363 23136 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
W20260923 15:09:10.748728 23135 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
I20260923 15:09:10.748809 23135 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce; no voters: bdf1565cf61e4f9b9ad8c4de288574af, e10ec0d212df4bcf8bdf401861092926
I20260923 15:09:10.748931 23518 raft_consensus.cc:2726] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:09:10.872951 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:10.872717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:45071)
0923 15:09:10.872856 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:10.872905 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260923 15:09:11.661564 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:11.661333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:54959)
0923 15:09:11.661474 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:11.661514 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":71,"threads_started":1}
I20260923 15:09:11.739058 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:11.738836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:44445)
0923 15:09:11.738975 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:11.739013 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":59,"threads_started":1}
I20260923 15:09:11.873718 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:11.873456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:33831)
0923 15:09:11.873618 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:11.873658 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":62,"threads_started":1}
I20260923 15:09:12.270730 23527 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:12.270836 23527 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:12.271128 23527 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
W20260923 15:09:12.271536 23135 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
W20260923 15:09:12.271618 23136 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
I20260923 15:09:12.271698 23136 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce; no voters: bdf1565cf61e4f9b9ad8c4de288574af, e10ec0d212df4bcf8bdf401861092926
I20260923 15:09:12.271790 23527 raft_consensus.cc:2726] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:09:12.662436 23528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:12.662124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:39443)
0923 15:09:12.662320 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:12.662372 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
I20260923 15:09:12.739773 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:12.739553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:48013)
0923 15:09:12.739693 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:12.739729 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260923 15:09:12.874464 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:12.874217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:60281)
0923 15:09:12.874374 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:12.874417 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
I20260923 15:09:13.663185 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:13.662950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:55649)
0923 15:09:13.663099 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:13.663139 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260923 15:09:13.740536 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:13.740319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:57969)
0923 15:09:13.740451 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:13.740492 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260923 15:09:13.875303 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:13.875067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:44535)
0923 15:09:13.875212 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:13.875256 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260923 15:09:14.547850 23534 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:14.547961 23534 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:14.548305 23534 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
W20260923 15:09:14.548812 23135 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
W20260923 15:09:14.548940 23136 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
I20260923 15:09:14.549021 23136 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce; no voters: bdf1565cf61e4f9b9ad8c4de288574af, e10ec0d212df4bcf8bdf401861092926
I20260923 15:09:14.549078 23534 raft_consensus.cc:2726] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:09:14.664371 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:14.664103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:56897)
0923 15:09:14.664248 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:14.664310 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260923 15:09:14.741415 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:14.741148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:40093)
0923 15:09:14.741302 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:14.741367 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260923 15:09:14.876107 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:14.875846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:50857)
0923 15:09:14.876010 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:14.876057 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260923 15:09:15.665354 23538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:15.665062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:45699)
0923 15:09:15.665253 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:15.665304 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260923 15:09:15.742452 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:15.742105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:39185)
0923 15:09:15.742323 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:15.742404 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":62,"threads_started":1}
I20260923 15:09:15.876945 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:15.876659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:52961)
0923 15:09:15.876814 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:15.876893 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
I20260923 15:09:16.666416 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:16.666028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:47261)
0923 15:09:16.666250 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:16.666365 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":163,"threads_started":1}
I20260923 15:09:16.743322 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:16.743049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:47547)
0923 15:09:16.743217 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:16.743269 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":67,"threads_started":1}
I20260923 15:09:16.877776 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:16.877510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:46743)
0923 15:09:16.877679 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:16.877724 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":79,"threads_started":1}
I20260923 15:09:17.667500 23544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:17.667073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:35463)
0923 15:09:17.667399 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:17.667448 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":229,"threads_started":1}
I20260923 15:09:17.744123 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:17.743909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:36969)
0923 15:09:17.744038 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:17.744075 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
I20260923 15:09:17.782393 23546 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:17.782501 23546 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:17.782681 23546 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
W20260923 15:09:17.783191 23136 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
W20260923 15:09:17.783306 23135 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
I20260923 15:09:17.783388 23135 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce; no voters: bdf1565cf61e4f9b9ad8c4de288574af, e10ec0d212df4bcf8bdf401861092926
I20260923 15:09:17.783469 23546 raft_consensus.cc:2726] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:09:17.878787 23547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:17.878448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:43293)
0923 15:09:17.878621 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:17.878734 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260923 15:09:18.668283 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:18.668058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:38983)
0923 15:09:18.668192 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:18.668227 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
I20260923 15:09:18.744954 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:18.744615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:55971)
0923 15:09:18.744809 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:18.744897 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260923 15:09:18.879688 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:18.879407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:43151)
0923 15:09:18.879567 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:18.879623 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260923 15:09:19.669070 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:19.668822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:38917)
0923 15:09:19.668976 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:19.669022 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260923 15:09:19.745803 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:19.745477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:44437)
0923 15:09:19.745643 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:19.745690 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260923 15:09:19.880577 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:19.880339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:49769)
0923 15:09:19.880485 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:19.880528 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260923 15:09:20.670065 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:20.669766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:56397)
0923 15:09:20.669936 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:20.670012 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":72,"threads_started":1}
I20260923 15:09:20.746536 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:20.746321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:39775)
0923 15:09:20.746447 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:20.746482 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260923 15:09:20.881501 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:20.881245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:37633)
0923 15:09:20.881400 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:20.881449 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260923 15:09:21.670886 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:21.670655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:33729)
0923 15:09:21.670780 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:21.670827 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260923 15:09:21.747181 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:21.746946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:34287)
0923 15:09:21.747095 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:21.747133 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260923 15:09:21.882405 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:21.882100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:49487)
0923 15:09:21.882239 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:21.882355 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":67,"threads_started":1}
I20260923 15:09:22.330461 23560 raft_consensus.cc:482] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:22.330592 23560 raft_consensus.cc:504] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdf1565cf61e4f9b9ad8c4de288574af" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41761 } } peers { permanent_uuid: "e10ec0d212df4bcf8bdf401861092926" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 38253 } } peers { permanent_uuid: "7d6494680a8b4118931c3b450db242ce" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43637 } }
I20260923 15:09:22.330884 23560 leader_election.cc:288] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761), e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253)
W20260923 15:09:22.331285 23135 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bdf1565cf61e4f9b9ad8c4de288574af (127.7.250.66:41761): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
W20260923 15:09:22.331367 23136 leader_election.cc:341] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e10ec0d212df4bcf8bdf401861092926 (127.7.250.67:38253): Not found: Tablet not found: 363686ebda1541e589d45d403d61ad97
I20260923 15:09:22.331420 23136 leader_election.cc:302] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d6494680a8b4118931c3b450db242ce; no voters: bdf1565cf61e4f9b9ad8c4de288574af, e10ec0d212df4bcf8bdf401861092926
I20260923 15:09:22.331502 23560 raft_consensus.cc:2726] T 363686ebda1541e589d45d403d61ad97 P 7d6494680a8b4118931c3b450db242ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:09:22.671908 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:22.671555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:32867)
0923 15:09:22.671784 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:22.671853 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260923 15:09:22.748034 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:22.747746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:34249)
0923 15:09:22.747941 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:22.747987 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":123,"threads_started":1}
I20260923 15:09:22.883369 23563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:22.883078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:37355)
0923 15:09:22.883243 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:22.883306 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":73,"threads_started":1}
I20260923 15:09:23.672838 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:23.672520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:46709)
0923 15:09:23.672742 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:23.672791 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":76,"threads_started":1}
I20260923 15:09:23.748922 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:23.748622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:57153)
0923 15:09:23.748810 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:23.748872 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":65,"threads_started":1}
I20260923 15:09:23.884199 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:23.883922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:54009)
0923 15:09:23.884080 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:23.884129 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
I20260923 15:09:24.673694 23567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:24.673462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.65:59757)
0923 15:09:24.673605 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:24.673647 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260923 15:09:24.749960 23568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:24.749697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.66:58945)
0923 15:09:24.749869 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:24.749913 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260923 15:09:24.885248 23569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:24.884932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34921 (local address 127.7.250.67:44739)
0923 15:09:24.885113 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:24.885198 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:34921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260923 15:09:24.885722 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23088
I20260923 15:09:24.890645 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23252
I20260923 15:09:24.895350 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23384
2026-09-23T15:09:24Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18604 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:09:24 dist-test-slave-bb3x krb5kdc[23572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:09:24 dist-test-slave-bb3x krb5kdc[23572](info): set up 2 sockets
Sep 23 15:09:24 dist-test-slave-bb3x krb5kdc[23572](info): commencing operation
krb5kdc: starting...
W20260923 15:09:26.000128 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:09:26.129884 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41811
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:26.212441 23585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:26.212672 23585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:26.212698 23585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:26.213855 23585 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 15:09:26.213897 23585 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 15:09:26.213909 23585 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 15:09:26.213922 23585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:26.213934 23585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:26.213948 23585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:26.213960 23585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:26.213971 23585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:26.215495 23585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:26.215721 23585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:26.215909 23585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:26.221403 23591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.221452 23585 server_base.cc:1034] running on GCE node
W20260923 15:09:26.221405 23590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:26.221454 23593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.221957 23585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:26.222375 23585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:26.223621 23585 hybrid_clock.cc:648] HybridClock initialized: now 1790176166223606 us; error 25 us; skew 500 ppm
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:26.232398 23585 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:26.232959 23585 webserver.cc:460] Webserver started at http://127.7.250.126:45449/ using document root <none> and password file <none>
I20260923 15:09:26.233237 23585 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:26.233292 23585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:26.233465 23585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:26.234690 23585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "beb90e1a8a0445438d8f30273411a297"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "52929738616e51a5557e0aeca7073c85"
server_key_iv: "57b4c4881924419cd8f29c299ef4e18f"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.235069 23585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "beb90e1a8a0445438d8f30273411a297"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "52929738616e51a5557e0aeca7073c85"
server_key_iv: "57b4c4881924419cd8f29c299ef4e18f"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.236503 23585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:26.237349 23600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.237608 23585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:26.237687 23585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "beb90e1a8a0445438d8f30273411a297"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "52929738616e51a5557e0aeca7073c85"
server_key_iv: "57b4c4881924419cd8f29c299ef4e18f"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.237784 23585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:26.247627 23585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:26.249702 23585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:26.249851 23585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:26.254077 23585 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41811
I20260923 15:09:26.254112 23652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41811 every 8 connection(s)
I20260923 15:09:26.254514 23585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:26.255159 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23585
I20260923 15:09:26.255272 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:09:26.255505 8169 external_mini_cluster.cc:1428] Setting key 78b8bd124b447b8f7f5420c68d2d16af
I20260923 15:09:26.255565 23653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:26.259491 23653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap starting.
I20260923 15:09:26.260252 23653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:26.260641 23653 log.cc:824] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:26.261463 23653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: No bootstrap required, opened a new log
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:26.264108 23653 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:26.264312 23653 raft_consensus.cc:374] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:26.264355 23653 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb90e1a8a0445438d8f30273411a297, State: Initialized, Role: FOLLOWER
I20260923 15:09:26.264496 23653 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:26.264605 23653 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:09:26.264663 23653 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:09:26.264724 23653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:26.265520 23653 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:26.265643 23653 leader_election.cc:302] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb90e1a8a0445438d8f30273411a297; no voters:
I20260923 15:09:26.265833 23653 leader_election.cc:288] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:09:26.265913 23658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:26.266072 23658 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 LEADER]: Becoming Leader. State: Replica: beb90e1a8a0445438d8f30273411a297, State: Running, Role: LEADER
I20260923 15:09:26.266166 23653 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:26.266209 23658 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:26.266734 23659 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beb90e1a8a0445438d8f30273411a297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } } }
I20260923 15:09:26.266824 23659 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:26.267060 23660 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb90e1a8a0445438d8f30273411a297. Latest consensus state: current_term: 1 leader_uuid: "beb90e1a8a0445438d8f30273411a297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } } }
I20260923 15:09:26.267414 23660 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:26.267613 23666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:26.268110 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.256091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43138 (local address 127.7.250.126:41811)
0923 15:09:26.256277 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.256290 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.256320 (+ 30us) server_negotiation.cc:407] Connection header received
0923 15:09:26.256742 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.256749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.256797 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.256909 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:26.257837 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.258850 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.260030 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.260517 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.260605 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.263456 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:26.263462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:26.263462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:26.263479 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:26.265690 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.266142 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.266146 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.266146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.266299 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.266564 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.266565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.266566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.266928 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:26.267638 (+ 710us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.267726 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.267913 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260923 15:09:26.268383 23666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:09:26.269958 23666 catalog_manager.cc:1347] Generated new cluster ID: 6953f49f84b94815ace5e159193337e3
I20260923 15:09:26.270012 23666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:09:26.324550 23666 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:09:26.325278 23666 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:09:26.336998 23666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Generated new TSK 0
I20260923 15:09:26.337256 23666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:09:26.374688 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41811
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:26.459682 23679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:26.459893 23679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:26.459919 23679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:26.461254 23679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:26.461303 23679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:26.461330 23679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:09:26.462886 23679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:26.463177 23679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:26.463440 23679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:26.468869 23685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:26.468876 23687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.469029 23679 server_base.cc:1034] running on GCE node
W20260923 15:09:26.469085 23684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.469374 23679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:26.469681 23679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:26.470881 23679 hybrid_clock.cc:648] HybridClock initialized: now 1790176166470845 us; error 54 us; skew 500 ppm
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:26.473837 23679 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:09:26.474406 23679 webserver.cc:460] Webserver started at http://127.7.250.65:43385/ using document root <none> and password file <none>
I20260923 15:09:26.474619 23679 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:26.474664 23679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:26.474773 23679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:26.475965 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "92d75fd1d58340f3b30dae2d4e578462"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "2dcab76acbfde40708dc33e7ab0fad58"
server_key_iv: "7595ef9456517a1739ed3975db2ea210"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.476305 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "92d75fd1d58340f3b30dae2d4e578462"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "2dcab76acbfde40708dc33e7ab0fad58"
server_key_iv: "7595ef9456517a1739ed3975db2ea210"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.477705 23679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:09:26.478508 23694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.478690 23679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:26.478804 23679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "92d75fd1d58340f3b30dae2d4e578462"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "2dcab76acbfde40708dc33e7ab0fad58"
server_key_iv: "7595ef9456517a1739ed3975db2ea210"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.478875 23679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:26.494907 23679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:26.497323 23679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:26.497445 23679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:26.497645 23679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:26.497972 23679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:26.498008 23679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.498032 23679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:26.498044 23679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.504493 23679 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:37401
I20260923 15:09:26.504565 23808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:37401 every 8 connection(s)
I20260923 15:09:26.504961 23679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:09:26.510509 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23679
I20260923 15:09:26.510636 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:09:26.510887 8169 external_mini_cluster.cc:1428] Setting key 07e09d40e1d7ce2d22f619cd81258772
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:26.515949 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.505808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:40001 (local address 127.7.250.126:41811)
0923 15:09:26.505884 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.505886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.506078 (+ 192us) server_negotiation.cc:407] Connection header received
0923 15:09:26.506715 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.506717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.506731 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.506792 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:26.507774 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.508344 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.509520 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.509765 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.509833 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.512407 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:26.512409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:26.512409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:26.512418 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:26.514150 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.514776 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.514777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.514777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.514848 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.515185 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.515185 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.515186 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.515309 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:26.515363 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.515785 (+ 422us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.515878 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:09:26.516276 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.505761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:40001)
0923 15:09:26.505942 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.505983 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.506556 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.506852 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.506855 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.507442 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:26.507654 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.507659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.508445 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.508446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.509423 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.509426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.509876 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.509877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.509994 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.510787 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:26.510800 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:26.512305 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:26.514250 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:26.514282 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:26.514285 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:26.514681 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:26.514932 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:26.514933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:26.514934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:26.515102 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:26.515381 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:26.515389 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:26.515591 (+ 202us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.515718 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.515955 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260923 15:09:26.517004 23809 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41811
I20260923 15:09:26.517154 23809 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:26.517395 23809 heartbeater.cc:511] Master 127.7.250.126:41811 requested a full tablet report, sending...
I20260923 15:09:26.518098 23617 ts_manager.cc:194] Registered new tserver with Master: 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65:37401)
I20260923 15:09:26.518797 23617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:40001
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:09:26.542456 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41811
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:26.629235 23814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:26.629478 23814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:26.629508 23814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:26.630865 23814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:26.630927 23814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:26.630959 23814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:09:26.633347 23814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:26.633677 23814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:26.633904 23814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:26.639757 23819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.640166 23814 server_base.cc:1034] running on GCE node
W20260923 15:09:26.640193 23820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:26.640197 23822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.640584 23814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:26.641000 23814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:26.642248 23814 hybrid_clock.cc:648] HybridClock initialized: now 1790176166642230 us; error 56 us; skew 500 ppm
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:26.645135 23814 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:09:26.645637 23814 webserver.cc:460] Webserver started at http://127.7.250.66:43335/ using document root <none> and password file <none>
I20260923 15:09:26.645841 23814 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:26.645886 23814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:26.645987 23814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:26.647119 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "4e390deaa72948b080d208019483fde2"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "7ff125e9da5d5d232744e2f08f005ae8"
server_key_iv: "7e2b38e972767a90d6c90d20077b6a91"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.647459 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "4e390deaa72948b080d208019483fde2"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "7ff125e9da5d5d232744e2f08f005ae8"
server_key_iv: "7e2b38e972767a90d6c90d20077b6a91"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.648931 23814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:09:26.649874 23829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.650141 23814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:26.650223 23814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "4e390deaa72948b080d208019483fde2"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "7ff125e9da5d5d232744e2f08f005ae8"
server_key_iv: "7e2b38e972767a90d6c90d20077b6a91"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.650323 23814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:26.669095 23814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:26.671456 23814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:26.671576 23814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:26.671780 23814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:26.672116 23814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:26.672151 23814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.672212 23814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:26.672245 23814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.678179 23814 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:42797
I20260923 15:09:26.678325 23942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:42797 every 8 connection(s)
I20260923 15:09:26.678642 23814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:26.688329 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23814
I20260923 15:09:26.688457 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:09:26.688745 8169 external_mini_cluster.cc:1428] Setting key 55db0fc3f07777090d6ec8daa52a70c2
I20260923 15:09:26.689811 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.679606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:40901 (local address 127.7.250.126:41811)
0923 15:09:26.679655 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.679658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.679921 (+ 263us) server_negotiation.cc:407] Connection header received
0923 15:09:26.680472 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.680474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.680486 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.680557 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:26.681756 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.682374 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.683527 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.683742 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.683819 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.686364 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:26.686366 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:26.686366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:26.686374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:26.687871 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.688466 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.688467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.688468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.688549 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.688945 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.688946 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.688946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.689160 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:26.689255 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.689617 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.689715 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:09:26.690120 23945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.679519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:40901)
0923 15:09:26.679742 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.679790 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.680349 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.680704 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.680708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.681342 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:26.681556 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.681572 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.682484 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.682486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.683421 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.683424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.683883 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.683884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.684011 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.684647 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:26.684656 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:26.686199 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:26.687969 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:26.687976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:26.687978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:26.688341 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:26.688630 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:26.688631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:26.688632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:26.688795 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:26.689247 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:26.689255 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:26.689449 (+ 194us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.689583 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.689879 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260923 15:09:26.690754 23943 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41811
I20260923 15:09:26.690876 23943 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:26.691112 23943 heartbeater.cc:511] Master 127.7.250.126:41811 requested a full tablet report, sending...
I20260923 15:09:26.691576 23617 ts_manager.cc:194] Registered new tserver with Master: 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
I20260923 15:09:26.692008 23617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:40901
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:09:26.718102 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41811
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:26.805006 23948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:26.805219 23948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:26.805244 23948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:26.806463 23948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:26.806509 23948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:26.806536 23948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:09:26.808055 23948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:26.808322 23948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:26.808566 23948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:26.814188 23954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:26.814180 23956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:26.814189 23953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:26.814412 23948 server_base.cc:1034] running on GCE node
I20260923 15:09:26.814888 23948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:26.815224 23948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:26.816408 23948 hybrid_clock.cc:648] HybridClock initialized: now 1790176166816386 us; error 41 us; skew 500 ppm
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:26.819286 23948 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:09:26.819995 23948 webserver.cc:460] Webserver started at http://127.7.250.67:45733/ using document root <none> and password file <none>
I20260923 15:09:26.820308 23948 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:26.820488 23948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:26.820672 23948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:09:26.821862 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "d5668227059c2b47e2ca9fb0965dfcef"
server_key_iv: "7e490ca0a2c09a8ed6d74abcc24f634b"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.822345 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "d5668227059c2b47e2ca9fb0965dfcef"
server_key_iv: "7e490ca0a2c09a8ed6d74abcc24f634b"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.824023 23948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:26.824983 23963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.825277 23948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:26.825387 23948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "d5668227059c2b47e2ca9fb0965dfcef"
server_key_iv: "7e490ca0a2c09a8ed6d74abcc24f634b"
server_key_version: "encryptionkey@0"
I20260923 15:09:26.825455 23948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:26.843578 23948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:26.845767 23948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:26.845898 23948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:26.846165 23948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:26.846534 23948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:26.846570 23948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.846592 23948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:26.846613 23948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:26.852586 23948 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:37901
I20260923 15:09:26.852777 24076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:37901 every 8 connection(s)
I20260923 15:09:26.853009 23948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:09:26.853829 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 23948
I20260923 15:09:26.853940 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:09:26.854149 8169 external_mini_cluster.cc:1428] Setting key ff4ca80d2fb6016dc8e0b59abc77d6c5
Sep 23 15:09:26 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176166, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:26.862828 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.853856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:51675 (local address 127.7.250.126:41811)
0923 15:09:26.853972 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.853974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.854061 (+ 87us) server_negotiation.cc:407] Connection header received
0923 15:09:26.854809 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.854810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.854823 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.854885 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:26.855918 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.856428 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.857438 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.857641 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.857730 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.860084 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:26.860085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:26.860086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:26.860094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:26.861369 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.861814 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.861815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.861816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.861876 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.862130 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.862131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.862131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.862304 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:26.862363 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.862643 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.862725 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 15:09:26.863054 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.853736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:51675)
0923 15:09:26.853917 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.853973 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.854712 (+ 739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.854960 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.854965 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.855608 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:26.855816 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.855821 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.856532 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.856533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.857347 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.857348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.857738 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.857739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.857890 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.858488 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:26.858496 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:26.859968 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:26.861475 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:26.861482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:26.861485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:26.861720 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:26.861943 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:26.861944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:26.861945 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:26.862054 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:26.862365 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:26.862372 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:26.862512 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.862623 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.862799 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260923 15:09:26.863828 24077 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41811
I20260923 15:09:26.863961 24077 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:26.864183 24077 heartbeater.cc:511] Master 127.7.250.126:41811 requested a full tablet report, sending...
I20260923 15:09:26.864744 23617 ts_manager.cc:194] Registered new tserver with Master: 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901)
I20260923 15:09:26.865175 23617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:51675
I20260923 15:09:26.866289 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:09:26.874009 23656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.867714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.7.250.126:41811)
0923 15:09:26.867796 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.867798 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.867844 (+ 46us) server_negotiation.cc:407] Connection header received
0923 15:09:26.867911 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.867912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.867924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.867999 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:26.868533 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.869072 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.869817 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.870050 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.870118 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.871490 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:26.871491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:26.871491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:26.871498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:26.872905 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.873299 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.873300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.873300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.873353 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:26.873508 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:26.873508 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:26.873509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:26.873646 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:26.873722 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.873885 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.873943 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:09:26.876191 23617 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:09:26.878062 23617 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:09:26.885516 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.881134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37901 (local address 127.0.0.1:53058)
0923 15:09:26.881829 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.881838 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.881886 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.882089 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.882090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.882092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:26.882140 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.882142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.883481 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.883482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.884454 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.884457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.885255 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.885256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.885323 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.885336 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.885402 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.885446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":6,"thread_start_us":109,"threads_started":1}
I20260923 15:09:26.885569 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.881479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53058 (local address 127.7.250.67:37901)
0923 15:09:26.881673 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.881677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.881950 (+ 273us) server_negotiation.cc:407] Connection header received
0923 15:09:26.881996 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.881998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.882013 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.882089 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:26.882208 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.883375 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.884547 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.885167 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.885240 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.885296 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.885434 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.885478 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260923 15:09:26.886066 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.881363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45644 (local address 127.7.250.66:42797)
0923 15:09:26.881560 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.881565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.881576 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:09:26.881621 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.881624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.881639 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.881730 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:26.882044 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.883107 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.884932 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.885665 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.885765 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.885821 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.885917 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.885972 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260923 15:09:26.886380 24090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.881132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42797 (local address 127.0.0.1:45644)
0923 15:09:26.881344 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.881389 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.881459 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.881741 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.881745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.881748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:26.881974 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.881977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.883229 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.883230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.884834 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.884837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.885767 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.885768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.885834 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.885838 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.885894 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.886308 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":62,"thread_start_us":70,"threads_started":1}
I20260923 15:09:26.886967 24011 tablet_service.cc:1467] Processing CreateTablet for tablet b50012a16a63431d8699b2b4c5271ae1 (DEFAULT_TABLE table=test-table [id=b1bbea32bdb04c079c6385d96b770215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:26.887084 23877 tablet_service.cc:1467] Processing CreateTablet for tablet b50012a16a63431d8699b2b4c5271ae1 (DEFAULT_TABLE table=test-table [id=b1bbea32bdb04c079c6385d96b770215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:26.887279 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.881134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37401 (local address 127.0.0.1:44298)
0923 15:09:26.881565 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.881574 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.881606 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.881831 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.881832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.881835 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:26.881897 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.881899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.883340 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.883340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.884835 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.884837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.886958 (+ 2121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.886959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.887034 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.887041 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.887150 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.887203 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":84,"thread_start_us":135,"threads_started":1}
I20260923 15:09:26.887320 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.881377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44298 (local address 127.7.250.65:37401)
0923 15:09:26.881584 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.881589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.881654 (+ 65us) server_negotiation.cc:407] Connection header received
0923 15:09:26.881705 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.881708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.881725 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.881798 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:26.881985 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.883224 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.885864 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.886712 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.886957 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.887033 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.887174 (+ 141us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.887221 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":88,"threads_started":1}
I20260923 15:09:26.887449 23877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50012a16a63431d8699b2b4c5271ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:26.887450 24011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50012a16a63431d8699b2b4c5271ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:26.891108 23742 tablet_service.cc:1467] Processing CreateTablet for tablet b50012a16a63431d8699b2b4c5271ae1 (DEFAULT_TABLE table=test-table [id=b1bbea32bdb04c079c6385d96b770215]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:26.891577 23742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b50012a16a63431d8699b2b4c5271ae1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:26.894512 24095 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Bootstrap starting.
I20260923 15:09:26.894560 24096 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap starting.
I20260923 15:09:26.895197 24096 tablet_bootstrap.cc:654] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:26.895383 24095 tablet_bootstrap.cc:654] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:26.895982 24095 log.cc:824] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:26.896311 24096 log.cc:824] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:26.896430 24094 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Bootstrap starting.
I20260923 15:09:26.896970 24095 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: No bootstrap required, opened a new log
I20260923 15:09:26.897110 24095 ts_tablet_manager.cc:1397] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:09:26.897202 24096 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: No bootstrap required, opened a new log
I20260923 15:09:26.897243 24094 tablet_bootstrap.cc:654] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:26.897400 24096 ts_tablet_manager.cc:1397] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:26.897934 24094 log.cc:824] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:26.898876 24094 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: No bootstrap required, opened a new log
I20260923 15:09:26.898958 24094 ts_tablet_manager.cc:1397] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:09:26.899422 24096 raft_consensus.cc:348] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.899616 24096 raft_consensus.cc:374] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:26.899684 24096 raft_consensus.cc:729] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Initialized, Role: FOLLOWER
I20260923 15:09:26.899855 24096 consensus_queue.cc:260] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.899776 24095 raft_consensus.cc:348] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.900146 24095 raft_consensus.cc:374] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:26.900278 24096 ts_tablet_manager.cc:1428] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:26.900285 23809 heartbeater.cc:503] Master 127.7.250.126:41811 was elected leader, sending a full tablet report...
I20260923 15:09:26.900329 24095 raft_consensus.cc:729] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bab6ca2a9b64aee94a11c0a541cfa59, State: Initialized, Role: FOLLOWER
I20260923 15:09:26.900480 24095 consensus_queue.cc:260] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.900885 24095 ts_tablet_manager.cc:1428] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 15:09:26.901100 24077 heartbeater.cc:503] Master 127.7.250.126:41811 was elected leader, sending a full tablet report...
I20260923 15:09:26.901392 24094 raft_consensus.cc:348] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.901563 24094 raft_consensus.cc:374] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:26.901588 24094 raft_consensus.cc:729] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e390deaa72948b080d208019483fde2, State: Initialized, Role: FOLLOWER
I20260923 15:09:26.901721 24094 consensus_queue.cc:260] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.902040 24094 ts_tablet_manager.cc:1428] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:26.902050 23943 heartbeater.cc:503] Master 127.7.250.126:41811 was elected leader, sending a full tablet report...
I20260923 15:09:26.924301 24100 raft_consensus.cc:482] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:26.924439 24100 raft_consensus.cc:504] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.924827 24100 leader_election.cc:288] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901), 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
I20260923 15:09:26.928117 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.925154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:41637 (local address 127.7.250.67:37901)
0923 15:09:26.925212 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.925214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.925265 (+ 51us) server_negotiation.cc:407] Connection header received
0923 15:09:26.925299 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.925300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.925310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.925389 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:26.925563 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.926128 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.927294 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.927745 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.927832 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.927842 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.928001 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.928045 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:09:26.928118 24091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.925268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:54617 (local address 127.7.250.66:42797)
0923 15:09:26.925324 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.925326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.925336 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:09:26.925430 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.925431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.925440 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.925508 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:26.925668 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.926145 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.927325 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.927755 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.927796 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.927803 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.928007 (+ 204us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.928049 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:09:26.928122 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.925081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37901 (local address 127.7.250.65:41637)
0923 15:09:26.925143 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.925157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.925204 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.925398 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.925400 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.925402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:26.925482 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.925484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.926232 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.926234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.927194 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.927207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.927829 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.927830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.927916 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.927920 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.927997 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.928044 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 15:09:26.928134 24103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.925081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42797 (local address 127.7.250.65:54617)
0923 15:09:26.925286 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.925297 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:26.925343 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:26.925511 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:26.925512 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:26.925514 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:26.925593 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.925594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.926251 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.926274 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.927216 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:26.927220 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.927854 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.927855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:26.927911 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.927914 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:09:26.927978 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 15:09:26.928026 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":16,"thread_start_us":121,"threads_started":1}
I20260923 15:09:26.928560 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50012a16a63431d8699b2b4c5271ae1" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e390deaa72948b080d208019483fde2" is_pre_election: true
I20260923 15:09:26.928674 24031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50012a16a63431d8699b2b4c5271ae1" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" is_pre_election: true
I20260923 15:09:26.928788 23897 raft_consensus.cc:2454] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 0.
I20260923 15:09:26.928862 24031 raft_consensus.cc:2454] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 0.
I20260923 15:09:26.929044 23698 leader_election.cc:302] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e390deaa72948b080d208019483fde2, 92d75fd1d58340f3b30dae2d4e578462; no voters:
I20260923 15:09:26.929229 24100 raft_consensus.cc:2781] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:09:26.929320 24100 raft_consensus.cc:482] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260923 15:09:26.929299 23944 tablet.cc:2369] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:09:26.929356 24100 raft_consensus.cc:3037] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:26.930226 24100 raft_consensus.cc:504] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.930421 24100 leader_election.cc:288] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 election: Requested vote from peers 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901), 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
I20260923 15:09:26.930600 23897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50012a16a63431d8699b2b4c5271ae1" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e390deaa72948b080d208019483fde2"
I20260923 15:09:26.930608 24031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b50012a16a63431d8699b2b4c5271ae1" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
I20260923 15:09:26.930688 24031 raft_consensus.cc:3037] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:26.930712 23897 raft_consensus.cc:3037] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:26.931447 23897 raft_consensus.cc:2454] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 1.
I20260923 15:09:26.931456 24031 raft_consensus.cc:2454] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 1.
I20260923 15:09:26.931627 23698 leader_election.cc:302] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e390deaa72948b080d208019483fde2, 92d75fd1d58340f3b30dae2d4e578462; no voters:
I20260923 15:09:26.931738 24100 raft_consensus.cc:2781] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:26.931921 24100 raft_consensus.cc:686] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 LEADER]: Becoming Leader. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Running, Role: LEADER
I20260923 15:09:26.932036 24100 consensus_queue.cc:237] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:26.932956 23617 catalog_manager.cc:5495] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "92d75fd1d58340f3b30dae2d4e578462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } health_report { overall_health: UNKNOWN } } }
I20260923 15:09:26.944710 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.942118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44308 (local address 127.7.250.65:37401)
0923 15:09:26.942186 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:26.942188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:26.942197 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:09:26.942238 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:26.942239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:26.942293 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:26.942357 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:26.942521 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.943007 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.943820 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:26.944038 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:26.944125 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:26.944316 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:26.944434 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:09:26.944533 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:26.944553 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 15:09:26.944574 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:09:26.946846 23897 raft_consensus.cc:1261] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Refusing update from remote peer 92d75fd1d58340f3b30dae2d4e578462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:26.946846 24031 raft_consensus.cc:1261] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Refusing update from remote peer 92d75fd1d58340f3b30dae2d4e578462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:26.947202 24100 consensus_queue.cc:1035] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:26.947336 24104 consensus_queue.cc:1035] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:26.950169 24107 mvcc.cc:205] Tried to move back new op lower bound from 7332561579810660352 to 7332561579754217472. Current Snapshot: MvccSnapshot[applied={T|T < 7332561579810660352}]
I20260923 15:09:26.950430 24109 mvcc.cc:205] Tried to move back new op lower bound from 7332561579810660352 to 7332561579754217472. Current Snapshot: MvccSnapshot[applied={T|T < 7332561579810660352}]
I20260923 15:09:26.952723 23616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:26.952857 23616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:26.953908 23616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Sending DeleteTablet for 3 replicas of tablet b50012a16a63431d8699b2b4c5271ae1
I20260923 15:09:26.954308 23742 tablet_service.cc:1514] Processing DeleteTablet for tablet b50012a16a63431d8699b2b4c5271ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:26 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:44298
I20260923 15:09:26.954443 24011 tablet_service.cc:1514] Processing DeleteTablet for tablet b50012a16a63431d8699b2b4c5271ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:26 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:53058
I20260923 15:09:26.954536 23877 tablet_service.cc:1514] Processing DeleteTablet for tablet b50012a16a63431d8699b2b4c5271ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:26 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:45644
I20260923 15:09:26.954766 24118 tablet_replica.cc:330] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: stopping tablet replica
I20260923 15:09:26.954872 24118 raft_consensus.cc:2229] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:26.954887 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23679
I20260923 15:09:26.954984 24118 raft_consensus.cc:2258] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:26.955111 24119 tablet_replica.cc:330] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: stopping tablet replica
I20260923 15:09:26.955628 24119 raft_consensus.cc:2229] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:26.955780 24119 raft_consensus.cc:2258] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:26.956283 24118 ts_tablet_manager.cc:1905] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:26.956286 24119 ts_tablet_manager.cc:1905] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:26.957971 24118 ts_tablet_manager.cc:1918] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:26.958069 24118 log.cc:1196] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/b50012a16a63431d8699b2b4c5271ae1
I20260923 15:09:26.958455 24118 ts_tablet_manager.cc:1939] T b50012a16a63431d8699b2b4c5271ae1 P 1bab6ca2a9b64aee94a11c0a541cfa59: Deleting consensus metadata
I20260923 15:09:26.958491 24119 ts_tablet_manager.cc:1918] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:26.958575 24119 log.cc:1196] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/b50012a16a63431d8699b2b4c5271ae1
I20260923 15:09:26.958899 24119 ts_tablet_manager.cc:1939] T b50012a16a63431d8699b2b4c5271ae1 P 4e390deaa72948b080d208019483fde2: Deleting consensus metadata
I20260923 15:09:26.958966 23603 catalog_manager.cc:4841] TS 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901): tablet b50012a16a63431d8699b2b4c5271ae1 (table test-table [id=b1bbea32bdb04c079c6385d96b770215]) successfully deleted
I20260923 15:09:26.959344 23604 catalog_manager.cc:4841] TS 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797): tablet b50012a16a63431d8699b2b4c5271ae1 (table test-table [id=b1bbea32bdb04c079c6385d96b770215]) successfully deleted
W20260923 15:09:26.962747 23602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:37401 (error 108)
I20260923 15:09:26.962930 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23814
I20260923 15:09:26.964027 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:26.963239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37401 (local address 127.0.0.1:44312)
0923 15:09:26.963950 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:26.963993 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":691}
W20260923 15:09:26.964174 23602 catalog_manager.cc:4584] TS 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65:37401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b50012a16a63431d8699b2b4c5271ae1: Network error: Client connection negotiation failed: client connection to 127.7.250.65:37401: connect: Connection refused (error 111)
I20260923 15:09:26.968755 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23948
I20260923 15:09:26.973975 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 23585
I20260923 15:09:46.979914 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41811
--webserver_interface=127.7.250.126
--webserver_port=45449
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:09:47.062960 24121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:47.063213 24121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:47.063262 24121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:47.064918 24121 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 15:09:47.064980 24121 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 15:09:47.065002 24121 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 15:09:47.065033 24121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:09:47.065052 24121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:47.065071 24121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:47.065092 24121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:09:47.065121 24121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:09:47.066880 24121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:47.067157 24121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:47.067438 24121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:47.076645 24127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:47.076647 24130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:47.076660 24128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:47.077003 24121 server_base.cc:1034] running on GCE node
I20260923 15:09:47.077281 24121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:47.077626 24121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:47.078823 24121 hybrid_clock.cc:648] HybridClock initialized: now 1790176187078799 us; error 36 us; skew 500 ppm
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:47.087421 24121 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:09:47.087934 24121 webserver.cc:460] Webserver started at http://127.7.250.126:45449/ using document root <none> and password file <none>
I20260923 15:09:47.088203 24121 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:47.088290 24121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:47.089825 24121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:47.090826 24137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.090996 24121 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:09:47.091096 24121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "beb90e1a8a0445438d8f30273411a297"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "52929738616e51a5557e0aeca7073c85"
server_key_iv: "57b4c4881924419cd8f29c299ef4e18f"
server_key_version: "encryptionkey@0"
I20260923 15:09:47.091529 24121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:47.107038 24121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:47.109573 24121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:47.109817 24121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:47.114445 24121 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41811
I20260923 15:09:47.114526 24189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41811 every 8 connection(s)
I20260923 15:09:47.114984 24121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:09:47.115672 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24121
I20260923 15:09:47.116045 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:37401
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:41811
--webserver_port=43385
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:47.116912 24190 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:09:47.117818 24190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap starting.
I20260923 15:09:47.127581 24190 log.cc:824] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:47.129937 24190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:09:47.130353 24190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap complete.
I20260923 15:09:47.132759 24190 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:47.133016 24190 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb90e1a8a0445438d8f30273411a297, State: Initialized, Role: FOLLOWER
I20260923 15:09:47.133194 24190 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:47.133270 24190 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:09:47.133340 24190 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:09:47.133394 24190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:09:47.134588 24190 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:47.134708 24190 leader_election.cc:302] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb90e1a8a0445438d8f30273411a297; no voters:
I20260923 15:09:47.134928 24190 leader_election.cc:288] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:09:47.134975 24194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:09:47.135105 24194 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 LEADER]: Becoming Leader. State: Replica: beb90e1a8a0445438d8f30273411a297, State: Running, Role: LEADER
I20260923 15:09:47.135213 24190 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:09:47.135195 24194 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:09:47.135505 24194 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb90e1a8a0445438d8f30273411a297. Latest consensus state: current_term: 2 leader_uuid: "beb90e1a8a0445438d8f30273411a297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } } }
I20260923 15:09:47.135602 24194 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:47.135766 24196 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "beb90e1a8a0445438d8f30273411a297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } } }
I20260923 15:09:47.135841 24196 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: This master's current role is: LEADER
I20260923 15:09:47.135960 24199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:09:47.136830 24199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:09:47.137290 24199 catalog_manager.cc:1259] Loaded cluster ID: 6953f49f84b94815ace5e159193337e3
I20260923 15:09:47.137439 24199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:09:47.138312 24199 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:09:47.138533 24199 catalog_manager.cc:5879] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Loaded TSK: 0
I20260923 15:09:47.146938 24199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Generated new TSK 1
I20260923 15:09:47.147095 24199 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:09:47.213382 24192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:47.213683 24192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:47.213729 24192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:47.215134 24192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:47.215217 24192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:47.215267 24192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:09:47.216935 24192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:37401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=43385
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:47.217252 24192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:47.217520 24192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:47.223287 24217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:47.223330 24192 server_base.cc:1034] running on GCE node
W20260923 15:09:47.223472 24218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:47.223277 24220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:47.224191 24192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:47.224540 24192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:47.225777 24192 hybrid_clock.cc:648] HybridClock initialized: now 1790176187225765 us; error 40 us; skew 500 ppm
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:47.229133 24192 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:09:47.229769 24192 webserver.cc:460] Webserver started at http://127.7.250.65:43385/ using document root <none> and password file <none>
I20260923 15:09:47.230051 24192 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:47.230141 24192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:47.232405 24192 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:47.233374 24227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.233660 24192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:47.233774 24192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "92d75fd1d58340f3b30dae2d4e578462"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "2dcab76acbfde40708dc33e7ab0fad58"
server_key_iv: "7595ef9456517a1739ed3975db2ea210"
server_key_version: "encryptionkey@0"
I20260923 15:09:47.234297 24192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:47.258921 24192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:47.262025 24192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:47.262223 24192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:47.262578 24192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:47.263195 24234 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:09:47.264899 24192 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:09:47.264974 24192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:09:47.265018 24192 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:09:47.265980 24192 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:09:47.266036 24192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:09:47.266201 24234 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap starting.
I20260923 15:09:47.274055 24192 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:37401
I20260923 15:09:47.274387 24341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:37401 every 8 connection(s)
I20260923 15:09:47.274605 24192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:09:47.278227 24234 log.cc:824] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:47.280965 24234 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:09:47.281307 24234 tablet_bootstrap.cc:492] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap complete.
I20260923 15:09:47.282645 24234 ts_tablet_manager.cc:1397] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.014s
I20260923 15:09:47.282939 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24192
I20260923 15:09:47.283214 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:42797
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:41811
--webserver_port=43335
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:47.284493 24234 raft_consensus.cc:348] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:47.284926 24234 raft_consensus.cc:729] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Initialized, Role: FOLLOWER
I20260923 15:09:47.285107 24234 consensus_queue.cc:260] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } } peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } }
I20260923 15:09:47.285422 24234 ts_tablet_manager.cc:1428] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260923 15:09:47.290086 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.275369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:55329)
0923 15:09:47.275579 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.275631 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.276310 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.276930 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.276935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.277346 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:47.277637 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.277646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.279485 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.279486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.280871 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.280876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.281864 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.281865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.282022 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.282761 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:47.282772 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:47.285051 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:47.287699 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.287709 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.287711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.288062 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.288448 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.288450 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.288450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.288618 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.289225 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:47.289235 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:47.289418 (+ 183us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.289537 (+ 119us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.289783 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260923 15:09:47.291124 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.275523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:55329 (local address 127.7.250.126:41811)
0923 15:09:47.275732 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.275741 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.275780 (+ 39us) server_negotiation.cc:407] Connection header received
0923 15:09:47.276600 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.276608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.276697 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.276866 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:47.277818 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.279315 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.280994 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.281725 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.281850 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.285290 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:47.285296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:47.285296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:47.285311 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:47.287583 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.288175 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.288180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.288180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.288331 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.288736 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.288736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.288737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.289114 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:47.290433 (+ 1319us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.290528 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.290899 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":73,"threads_started":1}
I20260923 15:09:47.291815 24342 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41811
I20260923 15:09:47.291944 24342 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:47.292171 24342 heartbeater.cc:511] Master 127.7.250.126:41811 requested a full tablet report, sending...
I20260923 15:09:47.293216 24154 ts_manager.cc:194] Registered new tserver with Master: 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65:37401)
I20260923 15:09:47.294552 24154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:55329
I20260923 15:09:47.295775 24342 heartbeater.cc:503] Master 127.7.250.126:41811 was elected leader, sending a full tablet report...
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:09:47.305603 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.293953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37401 (local address 127.0.0.1:52312)
0923 15:09:47.294342 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.294386 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.294468 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.295315 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.295321 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.295722 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:47.295796 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.295799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.297170 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.297171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.298155 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.298158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.298821 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.298822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.298909 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.299517 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:47.299523 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:47.301512 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:47.304084 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.304093 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.304096 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.304408 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.304685 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.304686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.304686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.304764 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.305140 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:47.305148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:47.305215 (+ 67us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.305335 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.305479 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":105,"threads_started":1}
I20260923 15:09:47.305624 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.294885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.7.250.65:37401)
0923 15:09:47.295103 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.295107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.295117 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:09:47.295155 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.295157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.295175 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.295248 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:47.295907 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.297068 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.298283 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.298736 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.298806 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.301645 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:47.301658 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:47.301658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:47.301668 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:47.303968 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.304520 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.304524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.304525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.304600 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.304841 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.304843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.304843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.305061 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:47.305155 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.305393 (+ 238us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.305515 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260923 15:09:47.306682 24276 tablet_service.cc:1514] Processing DeleteTablet for tablet b50012a16a63431d8699b2b4c5271ae1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:26 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:52312
I20260923 15:09:47.307273 24351 tablet_replica.cc:330] stopping tablet replica
I20260923 15:09:47.307404 24351 raft_consensus.cc:2229] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:47.307485 24351 raft_consensus.cc:2258] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:47.307880 24351 ts_tablet_manager.cc:1905] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:47.310963 24351 ts_tablet_manager.cc:1918] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:47.311061 24351 log.cc:1196] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal/wals/b50012a16a63431d8699b2b4c5271ae1
I20260923 15:09:47.311347 24351 ts_tablet_manager.cc:1939] T b50012a16a63431d8699b2b4c5271ae1 P 92d75fd1d58340f3b30dae2d4e578462: Deleting consensus metadata
I20260923 15:09:47.312007 24139 catalog_manager.cc:4841] TS 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65:37401): tablet b50012a16a63431d8699b2b4c5271ae1 (table test-table [id=b1bbea32bdb04c079c6385d96b770215]) successfully deleted
W20260923 15:09:47.377795 24347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:47.378041 24347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:47.378068 24347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:47.379359 24347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:47.379410 24347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:47.379439 24347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:09:47.380950 24347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:42797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=43335
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:47.381211 24347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:47.381501 24347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:47.387148 24356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:47.387107 24357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:47.387144 24347 server_base.cc:1034] running on GCE node
W20260923 15:09:47.387107 24359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:47.387632 24347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:47.387919 24347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:47.389092 24347 hybrid_clock.cc:648] HybridClock initialized: now 1790176187389074 us; error 27 us; skew 500 ppm
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:47.392114 24347 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:09:47.392706 24347 webserver.cc:460] Webserver started at http://127.7.250.66:43335/ using document root <none> and password file <none>
I20260923 15:09:47.392959 24347 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:47.393067 24347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:47.394873 24347 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:47.395853 24366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.396098 24347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:47.396291 24347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "4e390deaa72948b080d208019483fde2"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "7ff125e9da5d5d232744e2f08f005ae8"
server_key_iv: "7e2b38e972767a90d6c90d20077b6a91"
server_key_version: "encryptionkey@0"
I20260923 15:09:47.396713 24347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:47.422951 24347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:47.425452 24347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:47.425655 24347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:47.425966 24347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:47.426395 24347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:47.426457 24347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.426505 24347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:47.426574 24347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.433346 24347 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:42797
I20260923 15:09:47.433414 24479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:42797 every 8 connection(s)
I20260923 15:09:47.433779 24347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:09:47.440379 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24347
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:47.440680 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:37901
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:41811
--webserver_port=45733
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:09:47.444455 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.434611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:34209 (local address 127.7.250.126:41811)
0923 15:09:47.434661 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.434663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.434947 (+ 284us) server_negotiation.cc:407] Connection header received
0923 15:09:47.435538 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.435539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.435555 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.435616 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:47.436766 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.437378 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.438480 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.438697 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.438775 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.441216 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:47.441218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:47.441219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:47.441227 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:47.442823 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.443300 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.443301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.443301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.443370 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.443607 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.443608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.443608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.443776 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:47.443847 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.444195 (+ 348us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.444307 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:09:47.444661 24482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.434578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:34209)
0923 15:09:47.434819 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.434858 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.435420 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.435728 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.435732 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.436406 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:47.436608 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.436637 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.437483 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.437484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.438382 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.438384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.438787 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.438788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.438926 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.439505 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:47.439513 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:47.441108 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:47.442928 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.442935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.442937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.443211 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.443440 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.443440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.443440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.443543 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.443873 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:47.443881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:47.444021 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.444128 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.444399 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
I20260923 15:09:47.445164 24480 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41811
I20260923 15:09:47.445284 24480 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:47.445472 24480 heartbeater.cc:511] Master 127.7.250.126:41811 requested a full tablet report, sending...
I20260923 15:09:47.445946 24154 ts_manager.cc:194] Registered new tserver with Master: 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
I20260923 15:09:47.446437 24154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:34209
W20260923 15:09:47.523981 24483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:09:47.524259 24483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:09:47.524302 24483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:09:47.525606 24483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:09:47.525678 24483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:09:47.525728 24483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:09:47.527536 24483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:37901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=45733
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:09:47.527812 24483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:09:47.528064 24483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:09:47.533160 24488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:47.533174 24489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:09:47.533219 24491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:09:47.533273 24483 server_base.cc:1034] running on GCE node
I20260923 15:09:47.533708 24483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:09:47.533982 24483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:09:47.535149 24483 hybrid_clock.cc:648] HybridClock initialized: now 1790176187535146 us; error 37 us; skew 500 ppm
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:09:47.538064 24483 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:09:47.538787 24483 webserver.cc:460] Webserver started at http://127.7.250.67:45733/ using document root <none> and password file <none>
I20260923 15:09:47.539111 24483 fs_manager.cc:359] Metadata directory not provided
I20260923 15:09:47.539208 24483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:09:47.540714 24483 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:09:47.541461 24498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.541668 24483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:09:47.541805 24483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "d5668227059c2b47e2ca9fb0965dfcef"
server_key_iv: "7e490ca0a2c09a8ed6d74abcc24f634b"
server_key_version: "encryptionkey@0"
I20260923 15:09:47.542181 24483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:09:47.562718 24483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:09:47.565621 24483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:09:47.565780 24483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:09:47.566037 24483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:09:47.566479 24483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:09:47.566516 24483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.566540 24483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:09:47.566552 24483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:09:47.572566 24483 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:37901
I20260923 15:09:47.572645 24611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:37901 every 8 connection(s)
I20260923 15:09:47.573011 24483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:09:47.576699 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24483
Sep 23 15:09:47 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176187, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:09:47.584225 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.577230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49624 (local address 127.7.250.126:41811)
0923 15:09:47.577544 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.577546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.577552 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:09:47.577574 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.577574 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.577586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.577645 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:47.578309 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.578817 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.579615 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.579805 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.579867 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.581240 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:47.581241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:47.581242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:47.581249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:47.582873 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.583330 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.583331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.583331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.583402 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.583620 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.583621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.583621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.583739 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:47.583940 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.583979 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.584082 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":52,"threads_started":1}
I20260923 15:09:47.584306 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.573912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:34501 (local address 127.7.250.126:41811)
0923 15:09:47.574001 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.574004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.574104 (+ 100us) server_negotiation.cc:407] Connection header received
0923 15:09:47.574700 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.574701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.574713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.574787 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:47.575830 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.576402 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.577941 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.578152 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.578241 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.580800 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:47.580801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:47.580802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:47.580809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:47.582604 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.583155 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.583156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.583156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.583216 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.583508 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.583509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.583509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.583626 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:47.583700 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.584002 (+ 302us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.584165 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":55}
I20260923 15:09:47.584443 24614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.573818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:34501)
0923 15:09:47.573969 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.574016 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.574564 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.574870 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.574873 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.575509 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:09:47.575718 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.575724 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.576537 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.576538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.577851 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.577854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.578275 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.578276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.578419 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.578986 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:09:47.578995 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:09:47.580686 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:09:47.582690 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.582699 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.582701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.583077 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.583301 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:09:47.583302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:09:47.583303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:09:47.583426 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:09:47.583702 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:09:47.583711 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:09:47.583864 (+ 153us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.583981 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.584183 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260923 15:09:47.584856 24612 heartbeater.cc:347] Connected to a master server at 127.7.250.126:41811
I20260923 15:09:47.584964 24612 heartbeater.cc:464] Registering TS with master...
I20260923 15:09:47.585160 24612 heartbeater.cc:511] Master 127.7.250.126:41811 requested a full tablet report, sending...
I20260923 15:09:47.585523 24154 ts_manager.cc:194] Registered new tserver with Master: 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901)
I20260923 15:09:47.585851 24154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:34501
I20260923 15:09:47.596071 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:09:47.599192 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.596497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.7.250.126:41811)
0923 15:09:47.596587 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.596589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.596638 (+ 49us) server_negotiation.cc:407] Connection header received
0923 15:09:47.596717 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.596717 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.596755 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.596827 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:47.596967 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.597471 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.598231 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.598491 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.598548 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.598837 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:47.598891 (+ 54us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 15:09:47.599073 (+ 182us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49628: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260923 15:09:47.599288 24616 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49628: token expired
W20260923 15:09:47.599480 24082 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.250.126:41811)
I20260923 15:09:47.606189 24616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.599826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.7.250.126:41811)
0923 15:09:47.599880 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.599882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.599890 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:09:47.600009 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.600010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.600021 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.600067 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:09:47.600745 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.601230 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.601953 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.602163 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.602287 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.603573 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:09:47.603575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:09:47.603575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:09:47.603583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:09:47.604956 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.605374 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.605375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.605375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.605430 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:09:47.605650 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:09:47.605651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:09:47.605651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:09:47.605776 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:09:47.605891 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.605993 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.606081 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:09:47.607761 24154 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:09:47.609364 24154 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:09:47.611730 24276 tablet_service.cc:1467] Processing CreateTablet for tablet a95fe01dfe7648e8841e8bd0ea33dedb (DEFAULT_TABLE table=test-table [id=0fdc26dd0384495aa188b30dc2bab624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:47.611982 24276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95fe01dfe7648e8841e8bd0ea33dedb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:47.613515 24234 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap starting.
I20260923 15:09:47.613899 24234 tablet_bootstrap.cc:654] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:47.614521 24234 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: No bootstrap required, opened a new log
I20260923 15:09:47.614558 24234 ts_tablet_manager.cc:1397] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:47.614895 24234 raft_consensus.cc:348] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.614977 24234 raft_consensus.cc:374] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:47.615036 24234 raft_consensus.cc:729] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Initialized, Role: FOLLOWER
I20260923 15:09:47.615093 24234 consensus_queue.cc:260] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.615186 24234 ts_tablet_manager.cc:1428] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:09:47.616277 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.611446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37901 (local address 127.0.0.1:36468)
0923 15:09:47.611580 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.611595 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.611640 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.612233 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.612238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.612240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:47.612296 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.612298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.613465 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.613467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.614750 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.614754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.615934 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.615935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.616041 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.616067 (+ 26us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.616141 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.616199 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260923 15:09:47.616294 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.611536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.7.250.67:37901)
0923 15:09:47.612014 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.612018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.612035 (+ 17us) server_negotiation.cc:407] Connection header received
0923 15:09:47.612072 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.612091 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.612109 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.612198 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:47.612368 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.613365 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.614857 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.615738 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.615820 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.615902 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.616161 (+ 259us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.616212 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":112,"threads_started":1}
I20260923 15:09:47.617127 24546 tablet_service.cc:1467] Processing CreateTablet for tablet a95fe01dfe7648e8841e8bd0ea33dedb (DEFAULT_TABLE table=test-table [id=0fdc26dd0384495aa188b30dc2bab624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:47.617443 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.611496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42797 (local address 127.0.0.1:53290)
0923 15:09:47.613534 (+ 2038us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.613546 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.613607 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.613843 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.613845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.613848 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:47.613902 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.613903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.615176 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.615178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.616307 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.616310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.617180 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.617181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.617247 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.617251 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.617311 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.617358 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2022,"thread_start_us":66,"threads_started":1}
I20260923 15:09:47.617447 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.611752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.7.250.66:42797)
0923 15:09:47.611944 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.611950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.613675 (+ 1725us) server_negotiation.cc:407] Connection header received
0923 15:09:47.613725 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.613727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.613743 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.613822 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:47.613972 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.615044 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.616419 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.617085 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.617180 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.617238 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.617327 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.617366 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":114,"threads_started":1}
I20260923 15:09:47.617560 24546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95fe01dfe7648e8841e8bd0ea33dedb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:47.618196 24414 tablet_service.cc:1467] Processing CreateTablet for tablet a95fe01dfe7648e8841e8bd0ea33dedb (DEFAULT_TABLE table=test-table [id=0fdc26dd0384495aa188b30dc2bab624]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:09:47.618564 24414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95fe01dfe7648e8841e8bd0ea33dedb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:09:47.620803 24622 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Bootstrap starting.
I20260923 15:09:47.621134 24623 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Bootstrap starting.
I20260923 15:09:47.621658 24622 tablet_bootstrap.cc:654] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:47.621799 24623 tablet_bootstrap.cc:654] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Neither blocks nor log segments found. Creating new log.
I20260923 15:09:47.622408 24622 log.cc:824] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:47.622447 24623 log.cc:824] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:09:47.623356 24623 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: No bootstrap required, opened a new log
I20260923 15:09:47.623402 24622 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: No bootstrap required, opened a new log
I20260923 15:09:47.623446 24623 ts_tablet_manager.cc:1397] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:09:47.623471 24622 ts_tablet_manager.cc:1397] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:09:47.625511 24622 raft_consensus.cc:348] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.625511 24623 raft_consensus.cc:348] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.625715 24623 raft_consensus.cc:374] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:47.625715 24622 raft_consensus.cc:374] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:09:47.625756 24623 raft_consensus.cc:729] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e390deaa72948b080d208019483fde2, State: Initialized, Role: FOLLOWER
I20260923 15:09:47.625756 24622 raft_consensus.cc:729] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bab6ca2a9b64aee94a11c0a541cfa59, State: Initialized, Role: FOLLOWER
I20260923 15:09:47.625909 24622 consensus_queue.cc:260] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.625909 24623 consensus_queue.cc:260] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.626245 24623 ts_tablet_manager.cc:1428] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:47.626245 24622 ts_tablet_manager.cc:1428] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:09:47.626291 24612 heartbeater.cc:503] Master 127.7.250.126:41811 was elected leader, sending a full tablet report...
I20260923 15:09:47.626495 24480 heartbeater.cc:503] Master 127.7.250.126:41811 was elected leader, sending a full tablet report...
W20260923 15:09:47.684429 24481 tablet.cc:2369] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:09:47.775660 24343 tablet.cc:2369] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:09:47.785106 24348 raft_consensus.cc:482] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:09:47.785254 24348 raft_consensus.cc:504] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.785722 24348 leader_election.cc:288] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901), 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
I20260923 15:09:47.790324 24619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.786121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:46143 (local address 127.7.250.67:37901)
0923 15:09:47.786175 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.786177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.786211 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:09:47.786462 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.786464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.786473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.786531 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:47.786712 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.787408 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.789228 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.789866 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.789936 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.789945 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.790177 (+ 232us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.790222 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 15:09:47.790324 24620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.786076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:54947 (local address 127.7.250.66:42797)
0923 15:09:47.786125 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.786128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.786140 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:09:47.786288 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.786289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.786301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.786368 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:09:47.786609 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.787401 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.789228 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.789814 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.789936 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.789946 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.790177 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.790233 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:09:47.790352 24344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.785991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42797 (local address 127.7.250.65:54947)
0923 15:09:47.786064 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.786076 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.786141 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.786407 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.786408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.786410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:47.786505 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.786507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.787533 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.787534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.789106 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.789110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.789946 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.789947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.790046 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.790084 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.790158 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.790223 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":123}
I20260923 15:09:47.790367 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.785991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37901 (local address 127.7.250.65:46143)
0923 15:09:47.786155 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.786166 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:09:47.786234 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:09:47.786560 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:09:47.786561 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:09:47.786563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:09:47.786646 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.786647 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.787533 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.787534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.789106 (+ 1572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:09:47.789110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.789972 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.789973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:09:47.790046 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.790085 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 15:09:47.790157 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:09:47.790223 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":129,"thread_start_us":66,"threads_started":1}
I20260923 15:09:47.790876 24434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95fe01dfe7648e8841e8bd0ea33dedb" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e390deaa72948b080d208019483fde2" is_pre_election: true
I20260923 15:09:47.790867 24566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95fe01dfe7648e8841e8bd0ea33dedb" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" is_pre_election: true
I20260923 15:09:47.791051 24434 raft_consensus.cc:2454] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 0.
I20260923 15:09:47.791051 24566 raft_consensus.cc:2454] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 0.
I20260923 15:09:47.791386 24230 leader_election.cc:302] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bab6ca2a9b64aee94a11c0a541cfa59, 92d75fd1d58340f3b30dae2d4e578462; no voters:
I20260923 15:09:47.791579 24348 raft_consensus.cc:2781] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:09:47.791661 24348 raft_consensus.cc:482] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:09:47.791680 24348 raft_consensus.cc:3037] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:47.792378 24348 raft_consensus.cc:504] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.792546 24348 leader_election.cc:288] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 election: Requested vote from peers 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901), 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
I20260923 15:09:47.792714 24566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95fe01dfe7648e8841e8bd0ea33dedb" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
I20260923 15:09:47.792755 24434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95fe01dfe7648e8841e8bd0ea33dedb" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e390deaa72948b080d208019483fde2"
I20260923 15:09:47.792815 24566 raft_consensus.cc:3037] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:47.792827 24434 raft_consensus.cc:3037] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:09:47.793776 24566 raft_consensus.cc:2454] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 1.
I20260923 15:09:47.793776 24434 raft_consensus.cc:2454] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 1.
I20260923 15:09:47.794013 24231 leader_election.cc:302] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e390deaa72948b080d208019483fde2, 92d75fd1d58340f3b30dae2d4e578462; no voters:
I20260923 15:09:47.794127 24348 raft_consensus.cc:2781] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:09:47.794340 24348 raft_consensus.cc:686] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 LEADER]: Becoming Leader. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Running, Role: LEADER
I20260923 15:09:47.794446 24348 consensus_queue.cc:237] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:09:47.795272 24153 catalog_manager.cc:5495] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "92d75fd1d58340f3b30dae2d4e578462" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } health_report { overall_health: HEALTHY } } }
W20260923 15:09:47.823654 24613 tablet.cc:2369] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:09:47.846052 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.842823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.7.250.65:37401)
0923 15:09:47.842984 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 15:09:47.842986 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:09:47.842994 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:09:47.843033 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:09:47.843033 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:09:47.843053 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:09:47.843130 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:09:47.843273 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.843785 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.844860 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:09:47.845106 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:09:47.845180 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:09:47.845555 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:09:47.845741 (+ 186us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:09:47.845828 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:09:47.845899 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 15:09:47.845934 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260923 15:09:47.848070 24434 raft_consensus.cc:1261] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Refusing update from remote peer 92d75fd1d58340f3b30dae2d4e578462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:47.848242 24566 raft_consensus.cc:1261] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Refusing update from remote peer 92d75fd1d58340f3b30dae2d4e578462: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:09:47.848498 24629 consensus_queue.cc:1035] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:47.848663 24348 consensus_queue.cc:1035] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:09:47.851999 24634 mvcc.cc:205] Tried to move back new op lower bound from 7332561665422020608 to 7332561665206788096. Current Snapshot: MvccSnapshot[applied={T|T < 7332561665422020608}]
I20260923 15:09:47.853665 24154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:47.853783 24154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:09:47.854825 24154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Sending DeleteTablet for 3 replicas of tablet a95fe01dfe7648e8841e8bd0ea33dedb
I20260923 15:09:47.855165 24276 tablet_service.cc:1514] Processing DeleteTablet for tablet a95fe01dfe7648e8841e8bd0ea33dedb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:47 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:52312
I20260923 15:09:47.855180 24414 tablet_service.cc:1514] Processing DeleteTablet for tablet a95fe01dfe7648e8841e8bd0ea33dedb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:47 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:53290
I20260923 15:09:47.855449 24642 tablet_replica.cc:330] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: stopping tablet replica
I20260923 15:09:47.855515 24642 raft_consensus.cc:2229] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:09:47.855615 24642 raft_consensus.cc:2258] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:47.855603 24546 tablet_service.cc:1514] Processing DeleteTablet for tablet a95fe01dfe7648e8841e8bd0ea33dedb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:09:47 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:36468
I20260923 15:09:47.855887 24643 tablet_replica.cc:330] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: stopping tablet replica
I20260923 15:09:47.855988 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 24192
I20260923 15:09:47.856002 24643 raft_consensus.cc:2229] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:47.857084 24643 raft_consensus.cc:2258] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:47.857303 24641 tablet_replica.cc:330] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: stopping tablet replica
I20260923 15:09:47.857445 24641 raft_consensus.cc:2229] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:09:47.857563 24641 raft_consensus.cc:2258] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:09:47.857760 24643 ts_tablet_manager.cc:1905] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:09:47.860101 24643 ts_tablet_manager.cc:1918] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:09:47.860194 24643 log.cc:1196] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/a95fe01dfe7648e8841e8bd0ea33dedb
I20260923 15:09:47.860484 24643 ts_tablet_manager.cc:1939] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Deleting consensus metadata
I20260923 15:09:47.861509 24140 catalog_manager.cc:4841] TS 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901): tablet a95fe01dfe7648e8841e8bd0ea33dedb (table test-table [id=0fdc26dd0384495aa188b30dc2bab624]) successfully deleted
W20260923 15:09:47.862435 24139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:37401 (error 108)
I20260923 15:09:47.862967 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 24347
I20260923 15:09:47.863013 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.862819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37401 (local address 127.0.0.1:52334)
0923 15:09:47.862917 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.862974 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:37401: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260923 15:09:47.863140 24139 catalog_manager.cc:4584] TS 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65:37401): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a95fe01dfe7648e8841e8bd0ea33dedb: Network error: Client connection negotiation failed: client connection to 127.7.250.65:37401: connect: Connection refused (error 111)
I20260923 15:09:47.868256 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 24483
I20260923 15:09:47.868317 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:09:47.868187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42797 (local address 127.0.0.1:53294)
0923 15:09:47.868251 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:09:47.868283 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.66:42797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:09:47.873373 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 24121
I20260923 15:10:07.879033 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41811
--webserver_interface=127.7.250.126
--webserver_port=45449
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:07.965880 24645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:07.966113 24645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:07.966137 24645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:07.967455 24645 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 15:10:07.967501 24645 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 15:10:07.967514 24645 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 15:10:07.967525 24645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:07.967540 24645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:07.967552 24645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:07.967564 24645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:07.967576 24645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:07.969091 24645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=45449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:07.969314 24645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:07.969503 24645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:10:07.974989 24645 server_base.cc:1034] running on GCE node
W20260923 15:10:07.975167 24653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:07.975237 24650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:07.975193 24651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:07.975587 24645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:07.975898 24645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:07.977068 24645 hybrid_clock.cc:648] HybridClock initialized: now 1790176207977047 us; error 36 us; skew 500 ppm
Sep 23 15:10:07 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176207, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:07.984445 24645 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:07.984984 24645 webserver.cc:460] Webserver started at http://127.7.250.126:45449/ using document root <none> and password file <none>
I20260923 15:10:07.985265 24645 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:07.985337 24645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:07.987100 24645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:07.987857 24660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:07.988065 24645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:07.988179 24645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "beb90e1a8a0445438d8f30273411a297"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "52929738616e51a5557e0aeca7073c85"
server_key_iv: "57b4c4881924419cd8f29c299ef4e18f"
server_key_version: "encryptionkey@0"
I20260923 15:10:07.988580 24645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:08.003072 24645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:08.005668 24645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:08.005836 24645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:08.009757 24645 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41811
I20260923 15:10:08.009833 24713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41811 every 8 connection(s)
I20260923 15:10:08.010215 24645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:08.012033 24714 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:10:08.012806 24714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap starting.
I20260923 15:10:08.014788 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24645
I20260923 15:10:08.015182 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:37401
--local_ip_for_outbound_sockets=127.7.250.65
--tserver_master_addrs=127.7.250.126:41811
--webserver_port=43385
--webserver_interface=127.7.250.65
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:10:08.023612 24714 log.cc:824] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:08.026130 24714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:10:08.026566 24714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Bootstrap complete.
I20260923 15:10:08.029086 24714 raft_consensus.cc:348] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:10:08.029373 24714 raft_consensus.cc:729] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: beb90e1a8a0445438d8f30273411a297, State: Initialized, Role: FOLLOWER
I20260923 15:10:08.029613 24714 consensus_queue.cc:260] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:10:08.029713 24714 raft_consensus.cc:388] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:08.029773 24714 raft_consensus.cc:482] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:08.029830 24714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 2 FOLLOWER]: Advancing to term 3
I20260923 15:10:08.031056 24714 raft_consensus.cc:504] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:10:08.031188 24714 leader_election.cc:302] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beb90e1a8a0445438d8f30273411a297; no voters:
I20260923 15:10:08.031399 24714 leader_election.cc:288] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 15:10:08.031445 24718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 15:10:08.031584 24718 raft_consensus.cc:686] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [term 3 LEADER]: Becoming Leader. State: Replica: beb90e1a8a0445438d8f30273411a297, State: Running, Role: LEADER
I20260923 15:10:08.031684 24718 consensus_queue.cc:237] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } }
I20260923 15:10:08.031716 24714 sys_catalog.cc:564] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:08.032882 24719 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "beb90e1a8a0445438d8f30273411a297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } } }
I20260923 15:10:08.032971 24719 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:08.033273 24731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:08.033488 24720 sys_catalog.cc:455] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader beb90e1a8a0445438d8f30273411a297. Latest consensus state: current_term: 3 leader_uuid: "beb90e1a8a0445438d8f30273411a297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beb90e1a8a0445438d8f30273411a297" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41811 } } }
I20260923 15:10:08.033610 24720 sys_catalog.cc:458] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:08.034157 24731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:08.034283 24731 catalog_manager.cc:1259] Loaded cluster ID: 6953f49f84b94815ace5e159193337e3
I20260923 15:10:08.034317 24731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:08.035524 24731 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:08.035704 24731 catalog_manager.cc:5879] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Loaded TSK: 0
I20260923 15:10:08.035750 24731 catalog_manager.cc:5879] T 00000000000000000000000000000000 P beb90e1a8a0445438d8f30273411a297: Loaded TSK: 1
F20260923 15:10:08.036020 24731 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f4db3ef69ad google::LogMessage::Fail() at ??:0
@ 0x7f4db3ef988a google::LogMessage::SendToLog() at ??:0
@ 0x7f4db3ef64de google::LogMessage::Flush() at ??:0
@ 0x7f4db3ef7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4db6663104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f4db6660d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f4dbbe551cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f4dbbe5548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f4dbbe55594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4dbbe1bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4dbbe25ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f4db5dc40d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f4db5db9954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f4db6c1d6db start_thread at ??:0
@ 0x7f4db29f461f clone at ??:0
W20260923 15:10:08.123533 24716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:08.123762 24716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:08.123788 24716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:08.125176 24716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:08.125242 24716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:08.125275 24716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:08.127163 24716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:37401
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=43385
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:08.127453 24716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:08.127700 24716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:08.133477 24756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:08.133471 24755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:08.133757 24716 server_base.cc:1034] running on GCE node
W20260923 15:10:08.133471 24758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:08.134137 24716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:08.134471 24716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:08.135663 24716 hybrid_clock.cc:648] HybridClock initialized: now 1790176208135634 us; error 45 us; skew 500 ppm
Sep 23 15:10:08 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:08.138629 24716 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:10:08.139227 24716 webserver.cc:460] Webserver started at http://127.7.250.65:43385/ using document root <none> and password file <none>
I20260923 15:10:08.139487 24716 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:08.139560 24716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:08.141188 24716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:08.142230 24765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:08.142450 24716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:08.142601 24716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "92d75fd1d58340f3b30dae2d4e578462"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "2dcab76acbfde40708dc33e7ab0fad58"
server_key_iv: "7595ef9456517a1739ed3975db2ea210"
server_key_version: "encryptionkey@0"
I20260923 15:10:08.143190 24716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:08.155599 24716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:08.157985 24716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:08.158156 24716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:08.158440 24716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:08.158913 24772 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:10:08.161006 24716 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:10:08.161187 24716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:10:08.161237 24716 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:10:08.161947 24716 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:10:08.162004 24716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:10:08.162094 24772 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap starting.
I20260923 15:10:08.169541 24716 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:37401
I20260923 15:10:08.169708 24879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:37401 every 8 connection(s)
I20260923 15:10:08.169999 24716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:10:08.171844 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24716
I20260923 15:10:08.172127 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:42797
--local_ip_for_outbound_sockets=127.7.250.66
--tserver_master_addrs=127.7.250.126:41811
--webserver_port=43335
--webserver_interface=127.7.250.66
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:10:08.174007 24772 log.cc:824] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:08.177189 24772 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:10:08.177543 24772 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Bootstrap complete.
I20260923 15:10:08.178972 24772 ts_tablet_manager.cc:1397] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.011s
I20260923 15:10:08.180955 24772 raft_consensus.cc:348] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:08.181389 24772 raft_consensus.cc:729] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Initialized, Role: FOLLOWER
I20260923 15:10:08.181576 24772 consensus_queue.cc:260] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:08.181933 24772 ts_tablet_manager.cc:1428] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260923 15:10:08.256306 24882 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:08.170725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:45259)
0923 15:10:08.170910 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.170959 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:08.171467 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:08.256087 (+ 84620us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
W20260923 15:10:08.256683 24880 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:41811 (0 consecutive failures): Network error: Failed to ping master at 127.7.250.126:41811: Client connection negotiation failed: client connection to 127.7.250.126:41811: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:10:08.257082 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.256934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:50473)
0923 15:10:08.256991 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.257048 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 15:10:08.257485 24882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.257365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:40323)
0923 15:10:08.257417 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.257459 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:10:08.257653 24880 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:10:08.268671 24883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:08.268916 24883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:08.268971 24883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:08.270226 24883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:08.270336 24883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:08.270387 24883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:08.271987 24883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:42797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=43335
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:08.272279 24883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:08.272519 24883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:08.278136 24893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:08.278254 24890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:08.278273 24891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:08.278481 24883 server_base.cc:1034] running on GCE node
I20260923 15:10:08.278662 24883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:08.278926 24883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:08.280103 24883 hybrid_clock.cc:648] HybridClock initialized: now 1790176208280073 us; error 47 us; skew 500 ppm
Sep 23 15:10:08 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:08.282867 24883 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:10:08.283353 24883 webserver.cc:460] Webserver started at http://127.7.250.66:43335/ using document root <none> and password file <none>
I20260923 15:10:08.283627 24883 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:08.283701 24883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:08.285285 24883 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:08.285989 24900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:08.286244 24883 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:08.286388 24883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "4e390deaa72948b080d208019483fde2"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "7ff125e9da5d5d232744e2f08f005ae8"
server_key_iv: "7e2b38e972767a90d6c90d20077b6a91"
server_key_version: "encryptionkey@0"
I20260923 15:10:08.286790 24883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:08.300686 24883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:08.303224 24883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:08.303412 24883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:08.303680 24883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:08.304119 24907 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:10:08.305604 24883 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:10:08.305689 24883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:10:08.305742 24883 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:10:08.306510 24883 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:10:08.306565 24883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:08.306638 24907 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Bootstrap starting.
I20260923 15:10:08.313434 24883 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:42797
I20260923 15:10:08.313508 25014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:42797 every 8 connection(s)
I20260923 15:10:08.313875 24883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:10:08.315120 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.314583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:55123)
0923 15:10:08.314821 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.314908 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
W20260923 15:10:08.315346 25015 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:41811 (0 consecutive failures): Network error: Failed to ping master at 127.7.250.126:41811: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
I20260923 15:10:08.315650 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.315549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:32931)
0923 15:10:08.315600 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.315630 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:10:08.315972 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.315873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:46661)
0923 15:10:08.315919 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.315955 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:10:08.316092 25015 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:10:08.318462 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 24883
I20260923 15:10:08.318730 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:37901
--local_ip_for_outbound_sockets=127.7.250.67
--tserver_master_addrs=127.7.250.126:41811
--webserver_port=45733
--webserver_interface=127.7.250.67
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:10:08.366050 24907 log_util.cc:230] Ignoring log segment corruption in /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/a95fe01dfe7648e8841e8bd0ea33dedb.recovery/wal-000000001 because there are no log entries following the corrupted one. The server probably crashed in the middle of writing an entry to the write-ahead log or downloaded an active log via tablet copy. Error detail: Corruption: CRC mismatch in log entry header: Log file corruption detected. Failed trying to read batch #0 at offset 577 for log segment /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/a95fe01dfe7648e8841e8bd0ea33dedb.recovery/wal-000000001: Prior entries: [off=506 REPLICATE (1.1)] [off=506 REPLICATE (1.2)] [off=537 COMMIT (1.1)] [off=577 COMMIT (1.2)]
I20260923 15:10:08.366633 24907 log.cc:824] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:08.369463 24907 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:10:08.369819 24907 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Bootstrap complete.
I20260923 15:10:08.371411 24907 ts_tablet_manager.cc:1397] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Time spent bootstrapping tablet: real 0.065s user 0.052s sys 0.011s
I20260923 15:10:08.373165 24907 raft_consensus.cc:348] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:08.373517 24907 raft_consensus.cc:729] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e390deaa72948b080d208019483fde2, State: Initialized, Role: FOLLOWER
I20260923 15:10:08.373679 24907 consensus_queue.cc:260] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:08.373951 24907 ts_tablet_manager.cc:1428] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20260923 15:10:08.374346 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.374197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:45255)
0923 15:10:08.374279 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.374316 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:10:08.374688 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.374569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:36943)
0923 15:10:08.374620 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.374660 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:10:08.419703 25018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:08.419929 25018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:08.419957 25018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:08.421308 25018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:08.421365 25018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:08.421396 25018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:08.423058 25018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:37901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=45733
--tserver_master_addrs=127.7.250.126:41811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:08.423327 25018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:08.423559 25018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:08.429008 25028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:08.429092 25026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:08.429203 25025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:08.429062 25018 server_base.cc:1034] running on GCE node
I20260923 15:10:08.429519 25018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:08.429780 25018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:08.430919 25018 hybrid_clock.cc:648] HybridClock initialized: now 1790176208430901 us; error 36 us; skew 500 ppm
Sep 23 15:10:08 dist-test-slave-bb3x krb5kdc[23572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:08.433571 25018 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:10:08.434063 25018 webserver.cc:460] Webserver started at http://127.7.250.67:45733/ using document root <none> and password file <none>
I20260923 15:10:08.434369 25018 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:08.434440 25018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:08.435879 25018 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:08.436550 25035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:08.436883 25018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:08.436990 25018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "1bab6ca2a9b64aee94a11c0a541cfa59"
format_stamp: "Formatted at 2026-09-23 15:09:26 on dist-test-slave-bb3x"
server_key: "d5668227059c2b47e2ca9fb0965dfcef"
server_key_iv: "7e490ca0a2c09a8ed6d74abcc24f634b"
server_key_version: "encryptionkey@0"
I20260923 15:10:08.437410 25018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:08.446537 25018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:08.448565 25018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:08.448726 25018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:08.448952 25018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:08.449265 25018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:08.449317 25018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:08.449361 25018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:08.449411 25018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:08.455508 25018 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:37901
I20260923 15:10:08.455601 25148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:37901 every 8 connection(s)
I20260923 15:10:08.455935 25018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:10:08.457044 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.456658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:50673)
0923 15:10:08.456809 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.456884 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
W20260923 15:10:08.457477 25149 heartbeater.cc:644] Failed to heartbeat to 127.7.250.126:41811 (0 consecutive failures): Network error: Failed to ping master at 127.7.250.126:41811: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
I20260923 15:10:08.457767 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.457676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:45921)
0923 15:10:08.457720 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.457748 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 15:10:08.458005 25151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:08.457950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:35541)
0923 15:10:08.457972 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:08.457990 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260923 15:10:08.458119 25149 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:10:08.464684 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25018
I20260923 15:10:09.258494 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.258128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:50113)
0923 15:10:09.258355 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.258420 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":64,"threads_started":1}
I20260923 15:10:09.375515 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.375184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:52223)
0923 15:10:09.375395 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.375456 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":70,"threads_started":1}
I20260923 15:10:09.445362 25155 raft_consensus.cc:482] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:09.445545 25155 raft_consensus.cc:504] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:09.445926 25155 leader_election.cc:288] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901), 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
Sep 23 15:10:09 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.67@KRBTEST.COM
Sep 23 15:10:09 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.66@KRBTEST.COM
I20260923 15:10:09.458039 25157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.446289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:42243 (local address 127.7.250.67:37901)
0923 15:10:09.446467 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:09.446488 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:09.446508 (+ 20us) server_negotiation.cc:407] Connection header received
0923 15:10:09.446911 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:09.446913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:09.446956 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:09.447053 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:09.447759 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.449171 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.450739 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.451379 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.451948 (+ 569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:09.454021 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:09.454024 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:09.454025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:09.454034 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:09.456262 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:09.456831 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:09.456834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:09.456835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:09.456960 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:09.457216 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:09.457217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:09.457217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:09.457438 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:09.457530 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:09.457808 (+ 278us) server_negotiation.cc:299] Negotiation successful
0923 15:10:09.457932 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":67,"threads_started":1}
I20260923 15:10:09.458053 25156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.446137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42797 (local address 127.7.250.65:36131)
0923 15:10:09.446342 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.446353 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:09.446410 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:09.446830 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:09.446832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:09.447339 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:09.447611 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:09.447619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.449321 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.449322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:09.450634 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:09.450637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.451515 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.451517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:09.451617 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:09.452290 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:09.452298 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:09.454392 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:09.456334 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:09.456342 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:09.456345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:09.456745 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:09.457019 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:09.457020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:09.457021 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:09.457132 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:09.457533 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:09.457534 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:09.457653 (+ 119us) client_negotiation.cc:769] Sending connection context
0923 15:10:09.457749 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:10:09.457944 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":66,"thread_start_us":91,"threads_started":1}
I20260923 15:10:09.458241 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.446133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37901 (local address 127.7.250.65:42243)
0923 15:10:09.446235 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.446278 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:09.446341 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:09.447063 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:09.447064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:09.447351 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:09.447612 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:09.447619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.449319 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.449320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:09.450634 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:09.450637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.451489 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.451490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:09.451616 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:09.452337 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:09.452341 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:09.453908 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:09.456367 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:09.456367 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:09.456368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:09.456745 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:09.457047 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:09.457048 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:09.457048 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:09.457135 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:09.457508 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:09.457521 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:09.457653 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 15:10:09.457752 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 15:10:09.458023 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"mutex_wait_us":201}
I20260923 15:10:09.458706 25158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.446320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:36131 (local address 127.7.250.66:42797)
0923 15:10:09.446494 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:09.446503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:09.446521 (+ 18us) server_negotiation.cc:407] Connection header received
0923 15:10:09.446618 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:09.446620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:09.446659 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:09.446790 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:09.447751 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.449224 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.450739 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:09.451361 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:09.451482 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:09.454506 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:09.454510 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:09.454510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:09.454531 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:09.456225 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:09.456842 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:09.456845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:09.456845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:09.456955 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:09.457205 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:09.457206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:09.457206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:09.457447 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:09.457541 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:09.457831 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 15:10:09.457981 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":61,"threads_started":1}
I20260923 15:10:09.458730 25159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.458439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:60785)
0923 15:10:09.458639 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.458686 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":53,"threads_started":1}
W20260923 15:10:09.459088 24768 leader_election.cc:341] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901): Not found: Tablet not found: a95fe01dfe7648e8841e8bd0ea33dedb
I20260923 15:10:09.459192 24969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95fe01dfe7648e8841e8bd0ea33dedb" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4e390deaa72948b080d208019483fde2" is_pre_election: true
I20260923 15:10:09.459323 24969 raft_consensus.cc:2454] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 1.
I20260923 15:10:09.459532 24769 leader_election.cc:302] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e390deaa72948b080d208019483fde2, 92d75fd1d58340f3b30dae2d4e578462; no voters: 1bab6ca2a9b64aee94a11c0a541cfa59
I20260923 15:10:09.459642 25155 raft_consensus.cc:2781] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 15:10:09.459700 25155 raft_consensus.cc:482] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:09.459731 25155 raft_consensus.cc:3037] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:10:09.460788 25155 raft_consensus.cc:504] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:09.460927 25155 leader_election.cc:288] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 2 election: Requested vote from peers 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901), 4e390deaa72948b080d208019483fde2 (127.7.250.66:42797)
W20260923 15:10:09.461169 24768 leader_election.cc:341] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901): Not found: Tablet not found: a95fe01dfe7648e8841e8bd0ea33dedb
I20260923 15:10:09.461191 24969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95fe01dfe7648e8841e8bd0ea33dedb" candidate_uuid: "92d75fd1d58340f3b30dae2d4e578462" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4e390deaa72948b080d208019483fde2"
I20260923 15:10:09.461299 24969 raft_consensus.cc:3037] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:10:09.462641 24969 raft_consensus.cc:2454] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d75fd1d58340f3b30dae2d4e578462 in term 2.
I20260923 15:10:09.462908 24769 leader_election.cc:302] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e390deaa72948b080d208019483fde2, 92d75fd1d58340f3b30dae2d4e578462; no voters: 1bab6ca2a9b64aee94a11c0a541cfa59
I20260923 15:10:09.463018 25155 raft_consensus.cc:2781] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:10:09.463141 25155 raft_consensus.cc:686] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [term 2 LEADER]: Becoming Leader. State: Replica: 92d75fd1d58340f3b30dae2d4e578462, State: Running, Role: LEADER
I20260923 15:10:09.463230 25155 consensus_queue.cc:237] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:09.463550 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.463447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:57575)
0923 15:10:09.463492 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.463521 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 15:10:09.910723 24768 consensus_peers.cc:487] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 -> Peer 1bab6ca2a9b64aee94a11c0a541cfa59 (127.7.250.67:37901): Couldn't send request to peer 1bab6ca2a9b64aee94a11c0a541cfa59. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a95fe01dfe7648e8841e8bd0ea33dedb. This is attempt 1: this message will repeat every 5th retry.
I20260923 15:10:09.915262 24969 raft_consensus.cc:1261] T a95fe01dfe7648e8841e8bd0ea33dedb P 4e390deaa72948b080d208019483fde2 [term 2 FOLLOWER]: Refusing update from remote peer 92d75fd1d58340f3b30dae2d4e578462: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260923 15:10:09.915668 25155 consensus_queue.cc:1035] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260923 15:10:09.916767 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.916558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:36803)
0923 15:10:09.916666 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.916720 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":57,"threads_started":1}
I20260923 15:10:09.917529 25153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.917416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:53595)
0923 15:10:09.917465 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.917492 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260923 15:10:09.917850 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:09.917727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:38419)
0923 15:10:09.917796 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:09.917826 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:10:10.303973 25167 ts_tablet_manager.cc:927] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Initiating tablet copy from peer 92d75fd1d58340f3b30dae2d4e578462 (127.7.250.65:37401)
I20260923 15:10:10.304219 25167 tablet_copy_client.cc:316] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: tablet copy: Beginning tablet copy session from remote peer at address 127.7.250.65:37401
Sep 23 15:10:10 dist-test-slave-bb3x krb5kdc[23572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.65@KRBTEST.COM
I20260923 15:10:10.312005 25170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.304640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:57963 (local address 127.7.250.65:37401)
0923 15:10:10.304758 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:10.304762 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:10.304772 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:10:10.304798 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:10.304800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:10.304815 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:10.304901 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:10.305381 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:10.306042 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:10.306850 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:10.307173 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:10.307243 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:10.309294 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:10.309304 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:10.309304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:10.309311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:10.310985 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:10.311329 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:10.311332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:10.311332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:10.311386 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:10.311550 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:10.311550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:10.311550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:10.311688 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:10.311749 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:10.311869 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 15:10:10.311933 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":49,"threads_started":1}
I20260923 15:10:10.312022 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.304557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:37401 (local address 127.7.250.67:57963)
0923 15:10:10.304686 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:10.304702 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:10.304753 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:10.304942 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:10.304949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:10.305237 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:10.305309 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:10.305311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:10.306115 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:10.306116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:10.306768 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:10.306770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:10.307250 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:10.307251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:10.307317 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:10.307790 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:10.307794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:10.309216 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:10.311067 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:10.311073 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:10.311075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:10.311262 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:10.311438 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:10.311439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:10.311439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:10.311489 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:10.311739 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:10.311744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:10.311786 (+ 42us) client_negotiation.cc:769] Sending connection context
0923 15:10:10.311848 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 15:10:10.311935 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260923 15:10:10.312417 24854 tablet_copy_service.cc:140] P 92d75fd1d58340f3b30dae2d4e578462: Received BeginTabletCopySession request for tablet a95fe01dfe7648e8841e8bd0ea33dedb from peer 1bab6ca2a9b64aee94a11c0a541cfa59 ({username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:57963)
I20260923 15:10:10.312477 24854 tablet_copy_service.cc:161] P 92d75fd1d58340f3b30dae2d4e578462: Beginning new tablet copy session on tablet a95fe01dfe7648e8841e8bd0ea33dedb from peer 1bab6ca2a9b64aee94a11c0a541cfa59 at {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:57963: session id = 1bab6ca2a9b64aee94a11c0a541cfa59-a95fe01dfe7648e8841e8bd0ea33dedb
I20260923 15:10:10.312963 24854 tablet_copy_source_session.cc:205] T a95fe01dfe7648e8841e8bd0ea33dedb P 92d75fd1d58340f3b30dae2d4e578462: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 15:10:10.314033 25167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95fe01dfe7648e8841e8bd0ea33dedb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:10.317008 25167 tablet_copy_client.cc:791] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: tablet copy: Starting download of 0 data blocks...
I20260923 15:10:10.317166 25167 tablet_copy_client.cc:655] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: tablet copy: Starting download of 1 WAL segments...
I20260923 15:10:10.318058 25167 tablet_copy_client.cc:531] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 15:10:10.318830 25167 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Bootstrap starting.
I20260923 15:10:10.320420 25167 log.cc:824] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:10.323387 25167 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:10:10.323639 25167 tablet_bootstrap.cc:492] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Bootstrap complete.
I20260923 15:10:10.323784 25167 ts_tablet_manager.cc:1397] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260923 15:10:10.324780 25167 raft_consensus.cc:348] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:10.324931 25167 raft_consensus.cc:729] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bab6ca2a9b64aee94a11c0a541cfa59, State: Initialized, Role: FOLLOWER
I20260923 15:10:10.325068 25167 consensus_queue.cc:260] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bab6ca2a9b64aee94a11c0a541cfa59" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37901 } } peers { permanent_uuid: "4e390deaa72948b080d208019483fde2" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42797 } } peers { permanent_uuid: "92d75fd1d58340f3b30dae2d4e578462" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 37401 } }
I20260923 15:10:10.325346 25167 ts_tablet_manager.cc:1428] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:10:10.325663 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.325548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:54455)
0923 15:10:10.325600 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:10.325631 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 15:10:10.325686 24854 tablet_copy_service.cc:342] P 92d75fd1d58340f3b30dae2d4e578462: Request end of tablet copy session 1bab6ca2a9b64aee94a11c0a541cfa59-a95fe01dfe7648e8841e8bd0ea33dedb received from {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:57963
I20260923 15:10:10.325771 24854 tablet_copy_service.cc:433] P 92d75fd1d58340f3b30dae2d4e578462: ending tablet copy session 1bab6ca2a9b64aee94a11c0a541cfa59-a95fe01dfe7648e8841e8bd0ea33dedb on tablet a95fe01dfe7648e8841e8bd0ea33dedb with peer 1bab6ca2a9b64aee94a11c0a541cfa59
I20260923 15:10:10.325999 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.325899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:51345)
0923 15:10:10.325950 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:10.325978 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:10:10.902438 25174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.902217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:49471)
0923 15:10:10.902346 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:10.902381 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260923 15:10:10.902431 25103 raft_consensus.cc:1203] T a95fe01dfe7648e8841e8bd0ea33dedb P 1bab6ca2a9b64aee94a11c0a541cfa59 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260923 15:10:10.903074 25176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.902874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:53831)
0923 15:10:10.902979 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:10.903016 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":69,"threads_started":1}
I20260923 15:10:10.918553 25177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:10.918347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:44527)
0923 15:10:10.918456 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:10.918501 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":50,"threads_started":1}
I20260923 15:10:11.903223 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:11.902977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:33053)
0923 15:10:11.903131 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:11.903173 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260923 15:10:11.903720 25179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:11.903535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:46871)
0923 15:10:11.903649 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:11.903680 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":65,"threads_started":1}
I20260923 15:10:11.919322 25180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:11.919058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:49679)
0923 15:10:11.919218 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:11.919272 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":63,"threads_started":1}
I20260923 15:10:12.904053 25181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:12.903794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:43647)
0923 15:10:12.903932 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:12.904002 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260923 15:10:12.904383 25182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:12.904204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:36857)
0923 15:10:12.904300 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:12.904340 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":50,"threads_started":1}
I20260923 15:10:12.920120 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:12.919883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:43025)
0923 15:10:12.920024 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:12.920070 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260923 15:10:13.904788 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:13.904573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:44915)
0923 15:10:13.904702 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:13.904741 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260923 15:10:13.905077 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:13.904825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:60379)
0923 15:10:13.904969 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:13.905024 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260923 15:10:13.905290 25184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:13.905123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:50567)
0923 15:10:13.905219 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:13.905258 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260923 15:10:13.920993 25186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:13.920735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:36213)
0923 15:10:13.920887 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:13.920929 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":70,"threads_started":1}
I20260923 15:10:14.905784 25187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:14.905542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:57071)
0923 15:10:14.905695 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:14.905728 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260923 15:10:14.906129 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:14.905913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:50241)
0923 15:10:14.906040 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:14.906087 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260923 15:10:14.921904 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:14.921640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:34533)
0923 15:10:14.921807 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:14.921854 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
I20260923 15:10:15.906876 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:15.906566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:47771)
0923 15:10:15.906756 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:15.906810 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":89,"threads_started":1}
I20260923 15:10:15.907064 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:15.906766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:56433)
0923 15:10:15.906933 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:15.907008 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":71,"threads_started":1}
I20260923 15:10:15.922713 25192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:15.922463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:38981)
0923 15:10:15.922621 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:15.922664 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260923 15:10:16.907768 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:16.907479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:54825)
0923 15:10:16.907656 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:16.907708 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260923 15:10:16.907858 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:16.907553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:43005)
0923 15:10:16.907774 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:16.907814 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":103,"threads_started":1}
I20260923 15:10:16.923530 25195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:16.923252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:40879)
0923 15:10:16.923444 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:16.923481 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":116,"threads_started":1}
I20260923 15:10:17.908592 25196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:17.908356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:49113)
0923 15:10:17.908494 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:17.908541 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260923 15:10:17.908609 25197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:17.908396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:42959)
0923 15:10:17.908527 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:17.908565 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
I20260923 15:10:17.924355 25198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:17.924087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:49331)
0923 15:10:17.924220 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:17.924287 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260923 15:10:18.909556 25200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:18.909301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:45529)
0923 15:10:18.909474 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:18.909508 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260923 15:10:18.909556 25199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:18.909301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:41213)
0923 15:10:18.909465 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:18.909514 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260923 15:10:18.925289 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:18.924999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:58581)
0923 15:10:18.925191 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:18.925242 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":102,"threads_started":1}
I20260923 15:10:19.910327 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:19.910066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:33997)
0923 15:10:19.910198 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:19.910227 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260923 15:10:19.910372 25203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:19.910092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:51579)
0923 15:10:19.910240 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:19.910332 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":87,"threads_started":1}
I20260923 15:10:19.926054 25204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:19.925798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:45025)
0923 15:10:19.925968 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:19.926007 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260923 15:10:20.911085 25205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:20.910849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:51115)
0923 15:10:20.910996 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:20.911039 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":66,"threads_started":1}
I20260923 15:10:20.911100 25206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:20.910866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:51123)
0923 15:10:20.911021 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:20.911059 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260923 15:10:20.926935 25207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:20.926652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:37141)
0923 15:10:20.926843 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:20.926887 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":116,"threads_started":1}
I20260923 15:10:21.911988 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:21.911687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:39619)
0923 15:10:21.911896 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:21.911933 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":118,"threads_started":1}
I20260923 15:10:21.911988 25210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:21.911684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:49569)
0923 15:10:21.911868 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:21.911937 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":83,"threads_started":1}
I20260923 15:10:21.927759 25211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:21.927431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:52225)
0923 15:10:21.927582 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:21.927623 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":68,"threads_started":1}
I20260923 15:10:22.912722 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:22.912503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.67:59585)
0923 15:10:22.912637 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:22.912673 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260923 15:10:22.912760 25215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:22.912535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.65:39233)
0923 15:10:22.912680 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:22.912717 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
I20260923 15:10:22.928524 25216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:22.928298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41811 (local address 127.7.250.66:38043)
0923 15:10:22.928433 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:22.928474 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.126:41811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260923 15:10:23.468652 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 24716
I20260923 15:10:23.473475 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 24883
I20260923 15:10:23.477948 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25018
2026-09-23T15:10:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58582 ms)
[----------] 30 tests from SecurityITest (240177 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:23 dist-test-slave-bb3x krb5kdc[25219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:23 dist-test-slave-bb3x krb5kdc[25219](info): set up 2 sockets
Sep 23 15:10:23 dist-test-slave-bb3x krb5kdc[25219](info): commencing operation
krb5kdc: starting...
W20260923 15:10:24.544008 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:24 dist-test-slave-bb3x krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:24.681942 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34297
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:34297
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:24.765326 25232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:24.765578 25232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:24.765604 25232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:24.766862 25232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:24.766908 25232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:24.766923 25232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:24.766937 25232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:24.766948 25232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:24.768486 25232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:34297
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:34297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:24.768733 25232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:24.768923 25232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:24.774765 25238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:24.774796 25232 server_base.cc:1034] running on GCE node
W20260923 15:10:24.774936 25237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:24.774873 25240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:24.775374 25232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:24.775673 25232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:24.776860 25232 hybrid_clock.cc:648] HybridClock initialized: now 1790176224776818 us; error 50 us; skew 500 ppm
Sep 23 15:10:24 dist-test-slave-bb3x krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176224, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:24.779641 25232 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:24.780184 25232 webserver.cc:460] Webserver started at http://127.7.250.126:45983/ using document root <none> and password file <none>
I20260923 15:10:24.780429 25232 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:24.780499 25232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:24.780664 25232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:24.781787 25232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "27d8963ab27443e9ae362c995d7a043a"
format_stamp: "Formatted at 2026-09-23 15:10:24 on dist-test-slave-bb3x"
server_key: "23032a91a2c26dfa1ac353e693283eb0"
server_key_iv: "91f24455bae8d2de59cbdf076180c01f"
server_key_version: "encryptionkey@0"
I20260923 15:10:24.782179 25232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "27d8963ab27443e9ae362c995d7a043a"
format_stamp: "Formatted at 2026-09-23 15:10:24 on dist-test-slave-bb3x"
server_key: "23032a91a2c26dfa1ac353e693283eb0"
server_key_iv: "91f24455bae8d2de59cbdf076180c01f"
server_key_version: "encryptionkey@0"
I20260923 15:10:24.783759 25232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:24.784682 25247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:24.784932 25232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:24.785063 25232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "27d8963ab27443e9ae362c995d7a043a"
format_stamp: "Formatted at 2026-09-23 15:10:24 on dist-test-slave-bb3x"
server_key: "23032a91a2c26dfa1ac353e693283eb0"
server_key_iv: "91f24455bae8d2de59cbdf076180c01f"
server_key_version: "encryptionkey@0"
I20260923 15:10:24.785166 25232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:24.806133 25232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:24.808329 25232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:24.808506 25232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:24.813027 25232 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:34297
I20260923 15:10:24.813067 25299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:34297 every 8 connection(s)
I20260923 15:10:24.813484 25232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:24.814590 25300 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:24.817108 25300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: Bootstrap starting.
I20260923 15:10:24.817797 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25232
I20260923 15:10:24.817898 25300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:24.817934 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:24.818189 8169 external_mini_cluster.cc:1428] Setting key 092900bb88e847d030e979ccb902149a
I20260923 15:10:24.818293 25300 log.cc:824] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:24.819260 25300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: No bootstrap required, opened a new log
I20260923 15:10:24.821529 25300 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27d8963ab27443e9ae362c995d7a043a" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34297 } }
I20260923 15:10:24.821734 25300 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:24.821779 25300 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27d8963ab27443e9ae362c995d7a043a, State: Initialized, Role: FOLLOWER
I20260923 15:10:24.821942 25300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27d8963ab27443e9ae362c995d7a043a" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34297 } }
I20260923 15:10:24.822031 25300 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:24.822064 25300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:24.822111 25300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:24.822959 25300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27d8963ab27443e9ae362c995d7a043a" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34297 } }
I20260923 15:10:24.823077 25300 leader_election.cc:302] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27d8963ab27443e9ae362c995d7a043a; no voters:
I20260923 15:10:24.823251 25300 leader_election.cc:288] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:24.823323 25305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:24.823515 25300 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:24.823817 25305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [term 1 LEADER]: Becoming Leader. State: Replica: 27d8963ab27443e9ae362c995d7a043a, State: Running, Role: LEADER
I20260923 15:10:24.823984 25305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27d8963ab27443e9ae362c995d7a043a" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34297 } }
I20260923 15:10:24.824532 25306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27d8963ab27443e9ae362c995d7a043a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27d8963ab27443e9ae362c995d7a043a" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34297 } } }
I20260923 15:10:24.824623 25306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:24.824869 25307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27d8963ab27443e9ae362c995d7a043a. Latest consensus state: current_term: 1 leader_uuid: "27d8963ab27443e9ae362c995d7a043a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27d8963ab27443e9ae362c995d7a043a" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 34297 } } }
I20260923 15:10:24.824937 25307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a [sys.catalog]: This master's current role is: LEADER
W20260923 15:10:24.825534 25320 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:10:24.825647 25320 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:24.825770 25316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:24.825970 25316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:24.827880 25316 catalog_manager.cc:1347] Generated new cluster ID: 39a5d79a763244f0a5972988dddeb4aa
I20260923 15:10:24.827939 25316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:10:24 dist-test-slave-bb3x krb5kdc[25219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:24.833213 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:24.818779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.7.250.126:34297)
0923 15:10:24.818970 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:24.818980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:24.819018 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:10:24.820610 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:24.820620 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:24.820699 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:24.820866 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:24.821491 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:24.823442 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:24.824842 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:24.825649 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:24.826397 (+ 748us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:24.828829 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:24.828833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:24.828834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:24.828847 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:24.830994 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:24.831446 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:24.831449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:24.831449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:24.831554 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:24.831778 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:24.831779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:24.831779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:24.832091 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:24.832679 (+ 588us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:24.832784 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 15:10:24.832959 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260923 15:10:24.850534 25316 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:24.851336 25316 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:24.857475 25316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: Generated new TSK 0
I20260923 15:10:24.857717 25316 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:10:24.896099 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34297
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:24.986503 25326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:24.986726 25326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:24.986752 25326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:24.988034 25326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:24.988078 25326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:24.988107 25326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:24.989773 25326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:24.990031 25326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:24.990319 25326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:24.995878 25334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:24.995904 25331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:24.995893 25332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:24.996165 25326 server_base.cc:1034] running on GCE node
I20260923 15:10:24.996373 25326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:24.996647 25326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:24.997915 25326 hybrid_clock.cc:648] HybridClock initialized: now 1790176224997900 us; error 30 us; skew 500 ppm
Sep 23 15:10:24 dist-test-slave-bb3x krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176224, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:25.000671 25326 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:10:25.001176 25326 webserver.cc:460] Webserver started at http://127.7.250.65:45339/ using document root <none> and password file <none>
I20260923 15:10:25.001420 25326 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:25.001506 25326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:25.001660 25326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:25.002946 25326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "7c62bebd44b846d98bebacbbbded6c04"
format_stamp: "Formatted at 2026-09-23 15:10:24 on dist-test-slave-bb3x"
server_key: "c15e9bef01cdb06c657f272ca22f5af0"
server_key_iv: "8f4a7549fdbc6170a3cf57d34c546053"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.003325 25326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "7c62bebd44b846d98bebacbbbded6c04"
format_stamp: "Formatted at 2026-09-23 15:10:24 on dist-test-slave-bb3x"
server_key: "c15e9bef01cdb06c657f272ca22f5af0"
server_key_iv: "8f4a7549fdbc6170a3cf57d34c546053"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.004808 25326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:25.005682 25341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.005934 25326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:25.006037 25326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "7c62bebd44b846d98bebacbbbded6c04"
format_stamp: "Formatted at 2026-09-23 15:10:24 on dist-test-slave-bb3x"
server_key: "c15e9bef01cdb06c657f272ca22f5af0"
server_key_iv: "8f4a7549fdbc6170a3cf57d34c546053"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.006120 25326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:25.019943 25326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:25.022637 25326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:25.022811 25326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:25.023119 25326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:25.023486 25326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:25.023520 25326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.023566 25326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:25.023618 25326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.030504 25326 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:33461
I20260923 15:10:25.030545 25454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:33461 every 8 connection(s)
I20260923 15:10:25.030915 25326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:10:25.032138 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25326
I20260923 15:10:25.032270 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:10:25.032521 8169 external_mini_cluster.cc:1428] Setting key eb74b1c52be79a464f550d06880570da
Sep 23 15:10:25 dist-test-slave-bb3x krb5kdc[25219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176224, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
I20260923 15:10:25.043784 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.031666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:42585 (local address 127.7.250.126:34297)
0923 15:10:25.031818 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.031821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.032071 (+ 250us) server_negotiation.cc:407] Connection header received
0923 15:10:25.033281 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.033282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.033296 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.033351 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.034425 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.035054 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.036385 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.036636 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.036716 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.039866 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.039868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.039869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:25.039879 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.041764 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.042296 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.042297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.042298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.042366 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.042801 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.042802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.042803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.042987 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.043066 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.043461 (+ 395us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.043655 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260923 15:10:25.043967 25457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.031643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34297 (local address 127.7.250.65:42585)
0923 15:10:25.031865 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.031962 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.033193 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.033440 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.033445 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.033936 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:25.034314 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.034322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.035172 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.035173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.036269 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.036273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.036750 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.036751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.036910 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.037619 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:25.037631 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:25.039729 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:25.041878 (+ 2149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:25.041890 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:25.041892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:25.042164 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:25.042470 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:25.042471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:25.042471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:25.042634 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:25.043105 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:25.043112 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:25.043289 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.043404 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.043655 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":116,"threads_started":1}
I20260923 15:10:25.044698 25455 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34297
I20260923 15:10:25.044816 25455 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:25.044986 25455 heartbeater.cc:511] Master 127.7.250.126:34297 requested a full tablet report, sending...
I20260923 15:10:25.045635 25264 ts_manager.cc:194] Registered new tserver with Master: 7c62bebd44b846d98bebacbbbded6c04 (127.7.250.65:33461)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
I20260923 15:10:25.046497 25264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:42585
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:10:25.065450 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34297
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:25.151230 25460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:25.151436 25460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:25.151460 25460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:25.152527 25460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:25.152560 25460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:25.152586 25460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:25.153945 25460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:25.154130 25460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:25.154317 25460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:25.159515 25466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.159521 25465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:25.159628 25460 server_base.cc:1034] running on GCE node
W20260923 15:10:25.159521 25468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:25.160006 25460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:25.160288 25460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:25.161459 25460 hybrid_clock.cc:648] HybridClock initialized: now 1790176225161443 us; error 32 us; skew 500 ppm
Sep 23 15:10:25 dist-test-slave-bb3x krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:25.164093 25460 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:10:25.164628 25460 webserver.cc:460] Webserver started at http://127.7.250.66:46279/ using document root <none> and password file <none>
I20260923 15:10:25.164878 25460 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:25.164948 25460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:25.165110 25460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:25.166240 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "ab8e70f3c8cd4d948bdf223254a99ad4"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "9a8da965dbaf701890781ab06b939e18"
server_key_iv: "c3823327746a1e00a7a3cdc8667c47a2"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.166676 25460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "ab8e70f3c8cd4d948bdf223254a99ad4"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "9a8da965dbaf701890781ab06b939e18"
server_key_iv: "c3823327746a1e00a7a3cdc8667c47a2"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.168045 25460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:25.168833 25475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.169025 25460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:10:25.169124 25460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "ab8e70f3c8cd4d948bdf223254a99ad4"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "9a8da965dbaf701890781ab06b939e18"
server_key_iv: "c3823327746a1e00a7a3cdc8667c47a2"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.169243 25460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:25.212060 25460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:25.214496 25460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:25.214774 25460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:25.215051 25460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:25.215392 25460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:25.215456 25460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.215497 25460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:25.215541 25460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.221973 25460 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:46817
I20260923 15:10:25.222030 25588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:46817 every 8 connection(s)
I20260923 15:10:25.222385 25460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:10:25 dist-test-slave-bb3x krb5kdc[25219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:25.231607 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25460
I20260923 15:10:25.231755 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:10:25.232026 8169 external_mini_cluster.cc:1428] Setting key b0a7834ff1855a32ba52309a41b9b432
I20260923 15:10:25.233492 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.223129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:42993 (local address 127.7.250.126:34297)
0923 15:10:25.223174 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.223176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.223402 (+ 226us) server_negotiation.cc:407] Connection header received
0923 15:10:25.223981 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.223982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.223994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.224054 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.225117 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.225618 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.226894 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.227134 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.227217 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.229796 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.229798 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.229798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:25.229806 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.231389 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.232104 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.232106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.232106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.232209 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.232609 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.232610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.232610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.232772 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.232838 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.233353 (+ 515us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.233427 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:10:25.233866 25591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.223105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34297 (local address 127.7.250.66:42993)
0923 15:10:25.223263 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.223302 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.223875 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.224110 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.224113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.224805 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:25.225025 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.225031 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.225796 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.225797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.226773 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.226776 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.227248 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.227249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.227390 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.228127 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:25.228135 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:25.229662 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:25.231498 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:25.231508 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:25.231511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:25.231972 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:25.232375 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:25.232376 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:25.232376 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:25.232538 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:25.232984 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:25.232993 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:25.233190 (+ 197us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.233325 (+ 135us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.233531 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260923 15:10:25.234470 25589 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34297
I20260923 15:10:25.234588 25589 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:25.234877 25589 heartbeater.cc:511] Master 127.7.250.126:34297 requested a full tablet report, sending...
I20260923 15:10:25.235355 25264 ts_manager.cc:194] Registered new tserver with Master: ab8e70f3c8cd4d948bdf223254a99ad4 (127.7.250.66:46817)
I20260923 15:10:25.235839 25264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:42993
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:10:25.262962 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34297
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:45469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:25.346341 25594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:25.346561 25594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:25.346590 25594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:25.347816 25594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:25.347860 25594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:25.347888 25594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:25.349383 25594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:45469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:34297
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:25.349639 25594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:25.349879 25594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:25.355011 25599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.355032 25601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.355011 25603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:25.355212 25594 server_base.cc:1034] running on GCE node
I20260923 15:10:25.355613 25594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:25.355873 25594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:25.357084 25594 hybrid_clock.cc:648] HybridClock initialized: now 1790176225357067 us; error 31 us; skew 500 ppm
Sep 23 15:10:25 dist-test-slave-bb3x krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:25.360217 25594 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:10:25.360728 25594 webserver.cc:460] Webserver started at http://127.7.250.67:43877/ using document root <none> and password file <none>
I20260923 15:10:25.360961 25594 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:25.361032 25594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:25.361220 25594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:25.362444 25594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "752f3a337e8140d7a7e91c5a16dce3d5"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "bd9ef37420afffd2ebc9e3a7baa899ba"
server_key_iv: "b65cef6dfe8710ce195f7d3dc44ee36a"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.362818 25594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "752f3a337e8140d7a7e91c5a16dce3d5"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "bd9ef37420afffd2ebc9e3a7baa899ba"
server_key_iv: "b65cef6dfe8710ce195f7d3dc44ee36a"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.364277 25594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:10:25.365211 25610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.365430 25594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:25.365559 25594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "752f3a337e8140d7a7e91c5a16dce3d5"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "bd9ef37420afffd2ebc9e3a7baa899ba"
server_key_iv: "b65cef6dfe8710ce195f7d3dc44ee36a"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.365694 25594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:25.384789 25594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:25.387233 25594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:25.387539 25594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:25.387838 25594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:25.388275 25594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:25.388346 25594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.388399 25594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:25.388453 25594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.396034 25594 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:37215
I20260923 15:10:25.396087 25723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:37215 every 8 connection(s)
I20260923 15:10:25.396494 25594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:10:25.398159 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25594
I20260923 15:10:25.398307 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:10:25.398519 8169 external_mini_cluster.cc:1428] Setting key 97b4d95e0a85d5f8c1e3c98d9082b390
Sep 23 15:10:25 dist-test-slave-bb3x krb5kdc[25219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176225, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:25.406889 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.397340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:48979 (local address 127.7.250.126:34297)
0923 15:10:25.397426 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.397429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.397635 (+ 206us) server_negotiation.cc:407] Connection header received
0923 15:10:25.398343 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.398344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.398358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.398453 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.399781 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.400325 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.401288 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.401490 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.401548 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.403886 (+ 2338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.403887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.403888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:25.403894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.405378 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.405884 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.405885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.405885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.405937 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.406209 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.406210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.406210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.406368 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.406420 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.406749 (+ 329us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.406823 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:10:25.407012 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.397324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:34297 (local address 127.7.250.67:48979)
0923 15:10:25.397497 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.397540 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.398150 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.398555 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.398560 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.399252 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:25.399503 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.399508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.400415 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.400416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.401208 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.401210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.401593 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.401594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.401696 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.402360 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:25.402368 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:25.403773 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:25.405482 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:25.405490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:25.405492 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:25.405804 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:25.406013 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:25.406014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:25.406014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:25.406143 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:25.406433 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:25.406440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:25.406560 (+ 120us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.406681 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.406820 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260923 15:10:25.407512 25724 heartbeater.cc:347] Connected to a master server at 127.7.250.126:34297
I20260923 15:10:25.407613 25724 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:25.407814 25724 heartbeater.cc:511] Master 127.7.250.126:34297 requested a full tablet report, sending...
I20260923 15:10:25.408282 25264 ts_manager.cc:194] Registered new tserver with Master: 752f3a337e8140d7a7e91c5a16dce3d5 (127.7.250.67:37215)
I20260923 15:10:25.408740 25264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:48979
I20260923 15:10:25.410209 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:10:25.418210 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.411542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49098 (local address 127.7.250.126:34297)
0923 15:10:25.411607 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.411609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.411843 (+ 234us) server_negotiation.cc:407] Connection header received
0923 15:10:25.411868 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.411869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.411878 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.411930 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.412487 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.412992 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.413864 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.414048 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.414105 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.415434 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.415436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.415436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:25.415442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.416909 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.417276 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.417277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.417277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.417330 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:25.417638 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:25.417639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:25.417639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:25.417823 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.417911 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.418049 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.418131 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:10:25.420224 25264 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49098:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:10:25.422046 25264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:10:25.429308 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.425016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57252 (local address 127.7.250.66:46817)
0923 15:10:25.425388 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.425393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.425402 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:25.425442 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.425444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.425459 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.425803 (+ 344us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:25.425886 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.427047 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.428397 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.428884 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.428972 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.429034 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.429172 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.429217 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":133,"threads_started":1}
I20260923 15:10:25.429297 25737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.424756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46817 (local address 127.0.0.1:57252)
0923 15:10:25.425194 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.425205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.425263 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.425571 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.425574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.425577 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:25.425702 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.425705 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.427165 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.427166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.428312 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.428314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.428974 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.428975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.429067 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.429081 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.429152 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.429218 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":132,"thread_start_us":65,"threads_started":1}
I20260923 15:10:25.429410 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.425477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.7.250.65:33461)
0923 15:10:25.425678 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.425683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.425815 (+ 132us) server_negotiation.cc:407] Connection header received
0923 15:10:25.425864 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.425867 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.425883 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.425965 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:25.426079 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.427160 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.428250 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.428951 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.429002 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.429061 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.429277 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.429326 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260923 15:10:25.429410 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.425181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.7.250.67:37215)
0923 15:10:25.425337 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.425342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.425354 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:10:25.425394 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.425397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.425452 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.425806 (+ 354us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:25.425889 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.426736 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.428211 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.428955 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.429206 (+ 251us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.429273 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.429296 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.429337 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260923 15:10:25.429625 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.424756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:37215 (local address 127.0.0.1:56758)
0923 15:10:25.424962 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.425005 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.425068 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.425578 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.425579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.425582 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:25.425716 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.425719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.426835 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.426835 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.428113 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.428116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.429057 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.429058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.429127 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.429131 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.429161 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.429529 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":314,"thread_start_us":99,"threads_started":1}
I20260923 15:10:25.429634 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.424980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:33461 (local address 127.0.0.1:49786)
0923 15:10:25.425720 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.425731 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.425788 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.425977 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.425977 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.425979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:25.426013 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.426014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.427285 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.427286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.428125 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.428127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.429128 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.429128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.429185 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.429189 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.429488 (+ 299us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.429541 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":125,"threads_started":1}
I20260923 15:10:25.430655 25658 tablet_service.cc:1467] Processing CreateTablet for tablet f4a57e30f41b473a8b62f6fbbc4a893e (DEFAULT_TABLE table=test-table [id=a5d689c15b3c4dacbc77486184b51b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:25.430730 25389 tablet_service.cc:1467] Processing CreateTablet for tablet f4a57e30f41b473a8b62f6fbbc4a893e (DEFAULT_TABLE table=test-table [id=a5d689c15b3c4dacbc77486184b51b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:25.430832 25523 tablet_service.cc:1467] Processing CreateTablet for tablet f4a57e30f41b473a8b62f6fbbc4a893e (DEFAULT_TABLE table=test-table [id=a5d689c15b3c4dacbc77486184b51b28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:25.431025 25658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a57e30f41b473a8b62f6fbbc4a893e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:25.431093 25389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a57e30f41b473a8b62f6fbbc4a893e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:25.431103 25523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4a57e30f41b473a8b62f6fbbc4a893e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:25.433879 25741 tablet_bootstrap.cc:492] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Bootstrap starting.
I20260923 15:10:25.434374 25742 tablet_bootstrap.cc:492] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: Bootstrap starting.
I20260923 15:10:25.434602 25741 tablet_bootstrap.cc:654] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:25.435050 25742 tablet_bootstrap.cc:654] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:25.435211 25741 log.cc:824] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:25.435616 25742 log.cc:824] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:25.435940 25743 tablet_bootstrap.cc:492] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Bootstrap starting.
I20260923 15:10:25.436338 25742 tablet_bootstrap.cc:492] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: No bootstrap required, opened a new log
I20260923 15:10:25.436353 25741 tablet_bootstrap.cc:492] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: No bootstrap required, opened a new log
I20260923 15:10:25.436424 25741 ts_tablet_manager.cc:1397] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:10:25.436426 25742 ts_tablet_manager.cc:1397] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:25.436672 25743 tablet_bootstrap.cc:654] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:25.437214 25743 log.cc:824] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:25.437999 25743 tablet_bootstrap.cc:492] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: No bootstrap required, opened a new log
I20260923 15:10:25.438067 25743 ts_tablet_manager.cc:1397] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:25.438213 25742 raft_consensus.cc:348] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.438437 25742 raft_consensus.cc:374] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:25.438503 25742 raft_consensus.cc:729] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c62bebd44b846d98bebacbbbded6c04, State: Initialized, Role: FOLLOWER
I20260923 15:10:25.438632 25742 consensus_queue.cc:260] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.438556 25741 raft_consensus.cc:348] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.438719 25741 raft_consensus.cc:374] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:25.438750 25741 raft_consensus.cc:729] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 752f3a337e8140d7a7e91c5a16dce3d5, State: Initialized, Role: FOLLOWER
I20260923 15:10:25.438928 25742 ts_tablet_manager.cc:1428] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:10:25.438894 25741 consensus_queue.cc:260] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.439054 25455 heartbeater.cc:503] Master 127.7.250.126:34297 was elected leader, sending a full tablet report...
I20260923 15:10:25.439172 25741 ts_tablet_manager.cc:1428] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260923 15:10:25.439208 25724 heartbeater.cc:503] Master 127.7.250.126:34297 was elected leader, sending a full tablet report...
I20260923 15:10:25.440198 25743 raft_consensus.cc:348] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.440382 25743 raft_consensus.cc:374] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:25.440460 25743 raft_consensus.cc:729] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab8e70f3c8cd4d948bdf223254a99ad4, State: Initialized, Role: FOLLOWER
I20260923 15:10:25.440603 25743 consensus_queue.cc:260] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.440827 25743 ts_tablet_manager.cc:1428] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:25.440894 25589 heartbeater.cc:503] Master 127.7.250.126:34297 was elected leader, sending a full tablet report...
I20260923 15:10:25.458313 25748 raft_consensus.cc:482] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:25.458444 25748 raft_consensus.cc:504] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.458823 25748 leader_election.cc:288] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab8e70f3c8cd4d948bdf223254a99ad4 (127.7.250.66:46817), 7c62bebd44b846d98bebacbbbded6c04 (127.7.250.65:33461)
I20260923 15:10:25.461958 25738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.459093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:39219 (local address 127.7.250.66:46817)
0923 15:10:25.459165 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.459168 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.459177 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:25.459207 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.459208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.459220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.459284 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:25.459486 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.460156 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.461069 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.461452 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.461533 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.461542 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.461838 (+ 296us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.461878 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:10:25.461985 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.459240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:46765 (local address 127.7.250.65:33461)
0923 15:10:25.459282 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.459284 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.459383 (+ 99us) server_negotiation.cc:407] Connection header received
0923 15:10:25.459437 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.459438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.459447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.459496 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:25.459710 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.460335 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.461245 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.461613 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.461721 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.461729 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.461882 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.461918 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:10:25.462002 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.459003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46817 (local address 127.7.250.67:39219)
0923 15:10:25.459060 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.459073 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.459117 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.459328 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.459329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.459332 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:25.459404 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.459406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.460228 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.460229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.460974 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.460977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.461541 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.461542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.461625 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.461628 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.461830 (+ 202us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.461880 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:10:25.462024 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.459142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:33461 (local address 127.7.250.67:46765)
0923 15:10:25.459319 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.459329 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.459382 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.459496 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.459496 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.459498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:25.459568 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.459569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.460414 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.460415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.461147 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.461155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.461724 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.461725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.461796 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.461802 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.461875 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.461914 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":127,"threads_started":1}
I20260923 15:10:25.462441 25543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a57e30f41b473a8b62f6fbbc4a893e" candidate_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" is_pre_election: true
I20260923 15:10:25.462462 25409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a57e30f41b473a8b62f6fbbc4a893e" candidate_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c62bebd44b846d98bebacbbbded6c04" is_pre_election: true
I20260923 15:10:25.462590 25543 raft_consensus.cc:2454] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 752f3a337e8140d7a7e91c5a16dce3d5 in term 0.
I20260923 15:10:25.462680 25409 raft_consensus.cc:2454] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 752f3a337e8140d7a7e91c5a16dce3d5 in term 0.
I20260923 15:10:25.462780 25612 leader_election.cc:302] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 752f3a337e8140d7a7e91c5a16dce3d5, ab8e70f3c8cd4d948bdf223254a99ad4; no voters:
I20260923 15:10:25.462949 25748 raft_consensus.cc:2781] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:10:25.463034 25748 raft_consensus.cc:482] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:25.463070 25748 raft_consensus.cc:3037] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:25.463770 25748 raft_consensus.cc:504] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.463910 25748 leader_election.cc:288] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [CANDIDATE]: Term 1 election: Requested vote from peers ab8e70f3c8cd4d948bdf223254a99ad4 (127.7.250.66:46817), 7c62bebd44b846d98bebacbbbded6c04 (127.7.250.65:33461)
I20260923 15:10:25.464084 25543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a57e30f41b473a8b62f6fbbc4a893e" candidate_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4"
I20260923 15:10:25.464120 25409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4a57e30f41b473a8b62f6fbbc4a893e" candidate_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c62bebd44b846d98bebacbbbded6c04"
I20260923 15:10:25.464180 25543 raft_consensus.cc:3037] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:25.464198 25409 raft_consensus.cc:3037] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:25.464897 25409 raft_consensus.cc:2454] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 752f3a337e8140d7a7e91c5a16dce3d5 in term 1.
I20260923 15:10:25.464915 25543 raft_consensus.cc:2454] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 752f3a337e8140d7a7e91c5a16dce3d5 in term 1.
I20260923 15:10:25.465085 25612 leader_election.cc:302] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 752f3a337e8140d7a7e91c5a16dce3d5, 7c62bebd44b846d98bebacbbbded6c04; no voters:
I20260923 15:10:25.465206 25748 raft_consensus.cc:2781] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:25.465374 25748 raft_consensus.cc:686] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 1 LEADER]: Becoming Leader. State: Replica: 752f3a337e8140d7a7e91c5a16dce3d5, State: Running, Role: LEADER
I20260923 15:10:25.465478 25748 consensus_queue.cc:237] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } }
I20260923 15:10:25.466372 25264 catalog_manager.cc:5495] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 752f3a337e8140d7a7e91c5a16dce3d5 (127.7.250.67). New cstate: current_term: 1 leader_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "752f3a337e8140d7a7e91c5a16dce3d5" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 37215 } health_report { overall_health: HEALTHY } } }
W20260923 15:10:25.473047 25590 tablet.cc:2369] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:10:25.474105 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.471471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.7.250.67:37215)
0923 15:10:25.471506 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.471508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.471516 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:10:25.471581 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.471582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.471596 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.471715 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:25.471767 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.472303 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.473195 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.473400 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.473455 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.473732 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:25.473831 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:25.473908 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.474014 (+ 106us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.474036 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 15:10:25.475911 25409 raft_consensus.cc:1261] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 1 FOLLOWER]: Refusing update from remote peer 752f3a337e8140d7a7e91c5a16dce3d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:25.475929 25543 raft_consensus.cc:1261] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 1 FOLLOWER]: Refusing update from remote peer 752f3a337e8140d7a7e91c5a16dce3d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:25.476318 25751 consensus_queue.cc:1035] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c62bebd44b846d98bebacbbbded6c04" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 33461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:25.476450 25748 consensus_queue.cc:1035] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab8e70f3c8cd4d948bdf223254a99ad4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:25.479256 25755 mvcc.cc:205] Tried to move back new op lower bound from 7332561819546755072 to 7332561819507261440. Current Snapshot: MvccSnapshot[applied={T|T < 7332561819546755072}]
I20260923 15:10:25.479960 25756 mvcc.cc:205] Tried to move back new op lower bound from 7332561819546755072 to 7332561819507261440. Current Snapshot: MvccSnapshot[applied={T|T < 7332561819546755072}]
I20260923 15:10:25.481351 25264 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:25.481434 25264 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49098:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:25.482230 25264 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27d8963ab27443e9ae362c995d7a043a: Sending DeleteTablet for 3 replicas of tablet f4a57e30f41b473a8b62f6fbbc4a893e
I20260923 15:10:25.482621 25523 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a57e30f41b473a8b62f6fbbc4a893e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:25 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:57252
I20260923 15:10:25.482698 25389 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a57e30f41b473a8b62f6fbbc4a893e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:25 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:49786
I20260923 15:10:25.482625 25658 tablet_service.cc:1514] Processing DeleteTablet for tablet f4a57e30f41b473a8b62f6fbbc4a893e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:25 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:56758
I20260923 15:10:25.482975 25765 tablet_replica.cc:330] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04: stopping tablet replica
I20260923 15:10:25.483093 25765 raft_consensus.cc:2229] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:25.483203 25765 raft_consensus.cc:2258] T f4a57e30f41b473a8b62f6fbbc4a893e P 7c62bebd44b846d98bebacbbbded6c04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:25.483284 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25326
I20260923 15:10:25.487847 25764 tablet_replica.cc:330] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: stopping tablet replica
I20260923 15:10:25.487974 25764 raft_consensus.cc:2229] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:25.488085 25764 raft_consensus.cc:2258] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:25.488255 25766 tablet_replica.cc:330] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: stopping tablet replica
I20260923 15:10:25.488353 25766 raft_consensus.cc:2229] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:10:25.488447 25766 raft_consensus.cc:2258] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:25.488788 25764 ts_tablet_manager.cc:1905] T f4a57e30f41b473a8b62f6fbbc4a893e P ab8e70f3c8cd4d948bdf223254a99ad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:25.488976 25766 ts_tablet_manager.cc:1905] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:10:25.489095 25249 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:33461 (error 108)
I20260923 15:10:25.489878 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.489744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:33461 (local address 127.0.0.1:49794)
0923 15:10:25.489791 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.489841 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:33461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260923 15:10:25.490002 25249 catalog_manager.cc:4584] TS 7c62bebd44b846d98bebacbbbded6c04 (127.7.250.65:33461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f4a57e30f41b473a8b62f6fbbc4a893e: Network error: Client connection negotiation failed: client connection to 127.7.250.65:33461: connect: Connection refused (error 111)
I20260923 15:10:25.490177 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25460
I20260923 15:10:25.491451 25766 ts_tablet_manager.cc:1918] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:25.491518 25766 log.cc:1196] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/f4a57e30f41b473a8b62f6fbbc4a893e
I20260923 15:10:25.491799 25766 ts_tablet_manager.cc:1939] T f4a57e30f41b473a8b62f6fbbc4a893e P 752f3a337e8140d7a7e91c5a16dce3d5: Deleting consensus metadata
I20260923 15:10:25.492384 25250 catalog_manager.cc:4841] TS 752f3a337e8140d7a7e91c5a16dce3d5 (127.7.250.67:37215): tablet f4a57e30f41b473a8b62f6fbbc4a893e (table test-table [id=a5d689c15b3c4dacbc77486184b51b28]) successfully deleted
I20260923 15:10:25.495759 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25594
I20260923 15:10:25.495824 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.495701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46817 (local address 127.0.0.1:57268)
0923 15:10:25.495754 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.495787 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.66:46817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 15:10:25.500191 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25232
2026-09-23T15:10:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2021 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T15:10:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:25Z Disabled control of system clock
I20260923 15:10:25.529078 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40123
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:40123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:25.613348 25771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:25.613571 25771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:25.613593 25771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:25.614801 25771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:25.614847 25771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:25.614861 25771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:25.614873 25771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:25.614885 25771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:25.616235 25771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:40123
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:25.616448 25771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:25.616715 25771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:25.622061 25777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.622115 25779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.622066 25776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:25.622401 25771 server_base.cc:1034] running on GCE node
I20260923 15:10:25.622699 25771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:25.623059 25771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:25.624221 25771 hybrid_clock.cc:648] HybridClock initialized: now 1790176225624201 us; error 51 us; skew 500 ppm
I20260923 15:10:25.630707 25771 webserver.cc:460] Webserver started at http://127.7.250.126:36675/ using document root <none> and password file <none>
I20260923 15:10:25.631033 25771 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:25.631111 25771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:25.631276 25771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:25.632448 25771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "cd89a35ce978446790aaa01f6121bd4b"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "6f5b0611e77826b503c634f5b5be042c"
server_key_iv: "57109b3abf3b5cecddb6f36d062907db"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.632839 25771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "cd89a35ce978446790aaa01f6121bd4b"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "6f5b0611e77826b503c634f5b5be042c"
server_key_iv: "57109b3abf3b5cecddb6f36d062907db"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.634665 25771 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 15:10:25.635624 25785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.635901 25771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:25.636039 25771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "cd89a35ce978446790aaa01f6121bd4b"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "6f5b0611e77826b503c634f5b5be042c"
server_key_iv: "57109b3abf3b5cecddb6f36d062907db"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.636169 25771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:25.655190 25771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:25.655565 25771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:25.655746 25771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:25.660143 25771 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:40123
I20260923 15:10:25.660184 25837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:40123 every 8 connection(s)
I20260923 15:10:25.660563 25771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:25.661746 25838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:25.663730 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25771
I20260923 15:10:25.663818 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:25.664031 8169 external_mini_cluster.cc:1428] Setting key 45712c3bcd520c9f29ec1edf9f942e06
I20260923 15:10:25.664461 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b: Bootstrap starting.
I20260923 15:10:25.665202 25838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:25.665566 25838 log.cc:824] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:25.666446 25838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b: No bootstrap required, opened a new log
I20260923 15:10:25.668725 25838 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd89a35ce978446790aaa01f6121bd4b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40123 } }
I20260923 15:10:25.668907 25838 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:25.668941 25838 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd89a35ce978446790aaa01f6121bd4b, State: Initialized, Role: FOLLOWER
I20260923 15:10:25.669070 25838 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd89a35ce978446790aaa01f6121bd4b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40123 } }
I20260923 15:10:25.669163 25838 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:25.669195 25838 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:25.669270 25838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:25.669932 25838 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd89a35ce978446790aaa01f6121bd4b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40123 } }
I20260923 15:10:25.670042 25838 leader_election.cc:302] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd89a35ce978446790aaa01f6121bd4b; no voters:
I20260923 15:10:25.670233 25838 leader_election.cc:288] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:25.670356 25843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:25.670516 25843 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [term 1 LEADER]: Becoming Leader. State: Replica: cd89a35ce978446790aaa01f6121bd4b, State: Running, Role: LEADER
I20260923 15:10:25.670795 25843 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd89a35ce978446790aaa01f6121bd4b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40123 } }
I20260923 15:10:25.670647 25838 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:25.671021 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.664641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.7.250.126:40123)
0923 15:10:25.664885 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.664895 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.664935 (+ 40us) server_negotiation.cc:407] Connection header received
0923 15:10:25.665593 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.665602 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.665687 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.665849 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.666133 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.667546 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.668367 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.669012 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.669149 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.669930 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.669935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.669935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:25.670195 (+ 260us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.670282 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:25.670301 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.670373 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.670599 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.670697 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260923 15:10:25.671185 25844 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd89a35ce978446790aaa01f6121bd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd89a35ce978446790aaa01f6121bd4b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40123 } } }
I20260923 15:10:25.671197 25845 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd89a35ce978446790aaa01f6121bd4b. Latest consensus state: current_term: 1 leader_uuid: "cd89a35ce978446790aaa01f6121bd4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd89a35ce978446790aaa01f6121bd4b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40123 } } }
I20260923 15:10:25.671291 25844 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:25.671307 25845 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:25.671690 25847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:25.672217 25847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:25.674014 25847 catalog_manager.cc:1347] Generated new cluster ID: 961f4dae3001480c8999dfb0bff30886
I20260923 15:10:25.674089 25847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:25.681740 25847 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:25.682596 25847 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:25.687230 25847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b: Generated new TSK 0
I20260923 15:10:25.687477 25847 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:25.692304 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40123
--builtin_ntp_servers=127.7.250.84:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:25.776502 25862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:25.776728 25862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:25.776754 25862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:25.777997 25862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:25.778047 25862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:25.778077 25862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:25.779577 25862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:25.779834 25862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:25.780167 25862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:25.785634 25868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.785557 25870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.785557 25867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:25.786330 25862 server_base.cc:1034] running on GCE node
I20260923 15:10:25.786542 25862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:25.786862 25862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:25.788038 25862 hybrid_clock.cc:648] HybridClock initialized: now 1790176225788012 us; error 39 us; skew 500 ppm
I20260923 15:10:25.789570 25862 webserver.cc:460] Webserver started at http://127.7.250.65:35061/ using document root <none> and password file <none>
I20260923 15:10:25.789845 25862 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:25.789893 25862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:25.790074 25862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:25.791486 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "5c900e41db5f485aa663a626ec274253"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "f8b5c859db48558d6ecdaeb0bf05cb5e"
server_key_iv: "7780d29f15f27f2e751170c8fb366369"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.791921 25862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "5c900e41db5f485aa663a626ec274253"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "f8b5c859db48558d6ecdaeb0bf05cb5e"
server_key_iv: "7780d29f15f27f2e751170c8fb366369"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.794334 25862 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:25.795244 25876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.795476 25862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:25.795596 25862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "5c900e41db5f485aa663a626ec274253"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "f8b5c859db48558d6ecdaeb0bf05cb5e"
server_key_iv: "7780d29f15f27f2e751170c8fb366369"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.795678 25862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:25.808086 25862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:25.808377 25862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:25.808480 25862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:25.808682 25862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:25.809147 25862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:25.809209 25862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.809252 25862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:25.809283 25862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.815256 25862 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:35835
I20260923 15:10:25.815347 25989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:35835 every 8 connection(s)
I20260923 15:10:25.815709 25862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:10:25.817688 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25862
I20260923 15:10:25.817795 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:10:25.818018 8169 external_mini_cluster.cc:1428] Setting key d29fe273f1627fa744e7849a952fe174
I20260923 15:10:25.819191 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40123
--builtin_ntp_servers=127.7.250.84:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:10:25.822817 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.816580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:48983 (local address 127.7.250.126:40123)
0923 15:10:25.816653 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.816656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.816919 (+ 263us) server_negotiation.cc:407] Connection header received
0923 15:10:25.817481 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.817482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.817493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.817551 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.818545 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.819197 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.821009 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.821259 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.821347 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.822507 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.822508 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.822509 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:25.822515 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.822531 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:25.822536 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.822596 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.822692 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.822724 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:10:25.823148 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.816562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40123 (local address 127.7.250.65:48983)
0923 15:10:25.816786 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.816822 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.817329 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.817668 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.817673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.817684 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:25.818461 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.818469 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.819318 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.819319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.820880 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.820885 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.821374 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.821375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.821537 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.822234 (+ 697us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:25.822235 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:25.822347 (+ 112us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:25.822387 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:25.822602 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:25.822608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:25.822612 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.822693 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.822816 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260923 15:10:25.823793 25990 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40123
I20260923 15:10:25.823908 25990 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:25.824177 25990 heartbeater.cc:511] Master 127.7.250.126:40123 requested a full tablet report, sending...
I20260923 15:10:25.824851 25802 ts_manager.cc:194] Registered new tserver with Master: 5c900e41db5f485aa663a626ec274253 (127.7.250.65:35835)
I20260923 15:10:25.825907 25802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.65:48983
W20260923 15:10:25.903858 25993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:25.904119 25993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:25.904161 25993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:25.905396 25993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:25.905439 25993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:25.905465 25993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:25.906916 25993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:25.907115 25993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:25.907364 25993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:25.912657 25998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.912603 25999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:25.912624 26001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:25.912863 25993 server_base.cc:1034] running on GCE node
I20260923 15:10:25.913064 25993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:25.913281 25993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:25.914434 25993 hybrid_clock.cc:648] HybridClock initialized: now 1790176225914418 us; error 32 us; skew 500 ppm
I20260923 15:10:25.915844 25993 webserver.cc:460] Webserver started at http://127.7.250.66:42233/ using document root <none> and password file <none>
I20260923 15:10:25.916054 25993 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:25.916098 25993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:25.916194 25993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:25.917241 25993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "697f08ef37504af7ad641f8761f0f46a"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "985d2a46204a2c06f9cb81d619acde6e"
server_key_iv: "b60f1f9ab9a47b7972d2ec024d01253d"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.917544 25993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "697f08ef37504af7ad641f8761f0f46a"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "985d2a46204a2c06f9cb81d619acde6e"
server_key_iv: "b60f1f9ab9a47b7972d2ec024d01253d"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.919010 25993 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:10:25.919845 26007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.920039 25993 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:25.920164 25993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "697f08ef37504af7ad641f8761f0f46a"
format_stamp: "Formatted at 2026-09-23 15:10:25 on dist-test-slave-bb3x"
server_key: "985d2a46204a2c06f9cb81d619acde6e"
server_key_iv: "b60f1f9ab9a47b7972d2ec024d01253d"
server_key_version: "encryptionkey@0"
I20260923 15:10:25.920244 25993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:25.937141 25993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:25.937495 25993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:25.937668 25993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:25.937914 25993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:25.938421 25993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:25.938457 25993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.938479 25993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:25.938493 25993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:25.944434 25993 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:46041
I20260923 15:10:25.944494 26120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:46041 every 8 connection(s)
I20260923 15:10:25.944839 25993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:10:25.950562 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.945704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:33529 (local address 127.7.250.126:40123)
0923 15:10:25.945830 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:25.945832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:25.946033 (+ 201us) server_negotiation.cc:407] Connection header received
0923 15:10:25.946684 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:25.946685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:25.946699 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:25.946761 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:25.947509 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.948023 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.949034 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.949239 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.949335 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.950202 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:25.950203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:25.950203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:25.950209 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:25.950231 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:25.950235 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:25.950346 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:25.950447 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 15:10:25.950476 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260923 15:10:25.950791 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:25.945620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40123 (local address 127.7.250.66:33529)
0923 15:10:25.945904 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:25.945951 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:25.946569 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:25.946849 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:25.946853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:25.946861 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:25.947387 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.947391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.948126 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.948127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.948931 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:25.948933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:25.949353 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:25.949355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:25.949488 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:25.950046 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:25.950047 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:25.950078 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:25.950103 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:25.950350 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:25.950357 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:25.950360 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:25.950451 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:10:25.950558 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260923 15:10:25.951401 26121 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40123
I20260923 15:10:25.951493 26121 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:25.951644 26121 heartbeater.cc:511] Master 127.7.250.126:40123 requested a full tablet report, sending...
I20260923 15:10:25.952054 25802 ts_manager.cc:194] Registered new tserver with Master: 697f08ef37504af7ad641f8761f0f46a (127.7.250.66:46041)
I20260923 15:10:25.952417 25802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.66:33529
I20260923 15:10:25.955096 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 25993
I20260923 15:10:25.955181 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:10:25.955338 8169 external_mini_cluster.cc:1428] Setting key b277006c0a60062cd3e1abfc3386f444
I20260923 15:10:25.956569 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40123
--builtin_ntp_servers=127.7.250.84:46037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:26.039862 26124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:26.040066 26124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:26.040087 26124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:26.041265 26124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:26.041309 26124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:26.041334 26124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:26.042641 26124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:46037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40123
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:26.042827 26124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:26.043061 26124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:26.048311 26132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:26.048317 26129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:26.048334 26124 server_base.cc:1034] running on GCE node
W20260923 15:10:26.048337 26130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:26.049083 26124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:26.049739 26124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:26.050918 26124 hybrid_clock.cc:648] HybridClock initialized: now 1790176226050900 us; error 75 us; skew 500 ppm
I20260923 15:10:26.052335 26124 webserver.cc:460] Webserver started at http://127.7.250.67:45791/ using document root <none> and password file <none>
I20260923 15:10:26.052629 26124 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:26.052685 26124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:26.052826 26124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:26.054098 26124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b"
format_stamp: "Formatted at 2026-09-23 15:10:26 on dist-test-slave-bb3x"
server_key: "2b44077a243b9bd9bf20889ee566d44f"
server_key_iv: "b6a1e702e3220cc8a93495463253834f"
server_key_version: "encryptionkey@0"
I20260923 15:10:26.054556 26124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b"
format_stamp: "Formatted at 2026-09-23 15:10:26 on dist-test-slave-bb3x"
server_key: "2b44077a243b9bd9bf20889ee566d44f"
server_key_iv: "b6a1e702e3220cc8a93495463253834f"
server_key_version: "encryptionkey@0"
I20260923 15:10:26.056041 26124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:26.056887 26138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:26.057089 26124 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:26.057209 26124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b"
format_stamp: "Formatted at 2026-09-23 15:10:26 on dist-test-slave-bb3x"
server_key: "2b44077a243b9bd9bf20889ee566d44f"
server_key_iv: "b6a1e702e3220cc8a93495463253834f"
server_key_version: "encryptionkey@0"
I20260923 15:10:26.057293 26124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:26.097463 26124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:26.097829 26124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:26.097990 26124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:26.098245 26124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:26.098645 26124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:26.098703 26124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:26.098759 26124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:26.098791 26124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:26.104918 26124 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:41467
I20260923 15:10:26.104965 26251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:41467 every 8 connection(s)
I20260923 15:10:26.105345 26124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:10:26.111204 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.106321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:39771 (local address 127.7.250.126:40123)
0923 15:10:26.106429 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.106431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.106707 (+ 276us) server_negotiation.cc:407] Connection header received
0923 15:10:26.107246 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.107247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.107260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.107320 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:26.108061 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.108559 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.109650 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.109865 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.109943 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.110886 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:26.110888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:26.110888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:26.110894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:26.110917 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:26.110921 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:26.110985 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.111088 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.111120 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 15:10:26.111459 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.106248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40123 (local address 127.7.250.67:39771)
0923 15:10:26.106568 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.106606 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:26.107136 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:26.107395 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:26.107398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:26.107407 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:26.107930 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.107934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.108658 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.108659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.109552 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.109555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.109955 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.109956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.110096 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.110685 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:26.110686 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:26.110716 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:26.110741 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:26.110987 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:26.110994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:26.110997 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:26.111091 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 15:10:26.111198 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":127,"threads_started":1}
I20260923 15:10:26.112453 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26124
I20260923 15:10:26.112470 26252 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40123
I20260923 15:10:26.112577 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:10:26.112586 26252 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:26.112778 8169 external_mini_cluster.cc:1428] Setting key 016e2d500e11b1f3950aa2b4cf4cfe65
I20260923 15:10:26.112784 26252 heartbeater.cc:511] Master 127.7.250.126:40123 requested a full tablet report, sending...
I20260923 15:10:26.113179 25802 ts_manager.cc:194] Registered new tserver with Master: bc43bc44ee8e4347b1b0a7a98b92c38b (127.7.250.67:41467)
I20260923 15:10:26.113577 25802 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.67:39771
I20260923 15:10:26.113659 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:10:26.118616 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.115234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.7.250.126:40123)
0923 15:10:26.115326 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.115329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.115434 (+ 105us) server_negotiation.cc:407] Connection header received
0923 15:10:26.115519 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.115520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.115531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.115604 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:26.115807 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.116319 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.117114 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.117317 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.117403 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.118204 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:26.118205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:26.118205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:26.118211 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:26.118235 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:26.118239 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:26.118328 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.118517 (+ 189us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.118541 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 15:10:26.120385 25802 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47948:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:10:26.122080 25802 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:10:26.129184 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.125057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50746 (local address 127.7.250.67:41467)
0923 15:10:26.125393 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.125397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.125407 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:10:26.125446 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.125448 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.125462 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.125552 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:26.125838 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.127057 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.128274 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.128768 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.128864 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.128963 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.129049 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.129087 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":82,"threads_started":1}
I20260923 15:10:26.129196 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.124620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46041 (local address 127.0.0.1:60616)
0923 15:10:26.125094 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.125106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:26.125161 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:26.125566 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:26.125569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:26.125571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:26.125645 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.125647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.126703 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.126704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.127913 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.127915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.128853 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.128855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.128940 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.128957 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 15:10:26.129004 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:10:26.129070 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":143,"threads_started":1}
I20260923 15:10:26.129423 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.124609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:35835 (local address 127.0.0.1:46108)
0923 15:10:26.124749 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.124791 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:26.124857 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:26.125772 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:26.125773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:26.125776 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:26.125829 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.125830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.127171 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.127172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.128156 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.128159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.129077 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.129078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.129135 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.129138 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:26.129167 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 15:10:26.129204 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":135,"threads_started":1}
I20260923 15:10:26.129184 26264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.125057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:41467 (local address 127.0.0.1:50746)
0923 15:10:26.125207 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.125218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:26.125268 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:26.125730 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:26.125731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:26.125733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:26.125765 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.125766 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.127145 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.127146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.128180 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.128182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.128865 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.128867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.128942 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.128956 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:10:26.129034 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:10:26.129077 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":24,"thread_start_us":91,"threads_started":1}
I20260923 15:10:26.129494 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.125213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.7.250.66:46041)
0923 15:10:26.125379 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.125383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.125393 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:10:26.125433 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.125435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.125453 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.125541 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:26.125717 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.126606 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.128007 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.128748 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.129238 (+ 490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.129349 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.129379 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.129421 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
I20260923 15:10:26.130025 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.124877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46108 (local address 127.7.250.65:35835)
0923 15:10:26.125271 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.125277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.125618 (+ 341us) server_negotiation.cc:407] Connection header received
0923 15:10:26.125662 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.125665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.125681 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.125761 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:26.125891 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.127058 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.128258 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.128970 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.129241 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.129883 (+ 642us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.129908 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.129945 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":115,"threads_started":1}
I20260923 15:10:26.130179 26055 tablet_service.cc:1467] Processing CreateTablet for tablet 903a9701941948229c709aafcb49bab4 (DEFAULT_TABLE table=test-table [id=cc077e61c95e48b88d0dba637b1abc56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:26.130636 26055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903a9701941948229c709aafcb49bab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:26.130730 25924 tablet_service.cc:1467] Processing CreateTablet for tablet 903a9701941948229c709aafcb49bab4 (DEFAULT_TABLE table=test-table [id=cc077e61c95e48b88d0dba637b1abc56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:26.130834 26186 tablet_service.cc:1467] Processing CreateTablet for tablet 903a9701941948229c709aafcb49bab4 (DEFAULT_TABLE table=test-table [id=cc077e61c95e48b88d0dba637b1abc56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:26.131050 25924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903a9701941948229c709aafcb49bab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:26.131203 26186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903a9701941948229c709aafcb49bab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:26.133826 26266 tablet_bootstrap.cc:492] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Bootstrap starting.
I20260923 15:10:26.134240 26267 tablet_bootstrap.cc:492] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Bootstrap starting.
I20260923 15:10:26.134392 26268 tablet_bootstrap.cc:492] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: Bootstrap starting.
I20260923 15:10:26.134724 26266 tablet_bootstrap.cc:654] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:26.134930 26267 tablet_bootstrap.cc:654] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:26.134995 26268 tablet_bootstrap.cc:654] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:26.135095 26266 log.cc:824] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:26.135279 26267 log.cc:824] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:26.135298 26268 log.cc:824] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:26.135977 26266 tablet_bootstrap.cc:492] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: No bootstrap required, opened a new log
I20260923 15:10:26.136049 26266 ts_tablet_manager.cc:1397] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:26.136300 26267 tablet_bootstrap.cc:492] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: No bootstrap required, opened a new log
I20260923 15:10:26.136368 26267 ts_tablet_manager.cc:1397] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:26.138768 26266 raft_consensus.cc:348] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.138856 26267 raft_consensus.cc:348] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.138947 26266 raft_consensus.cc:374] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:26.138980 26267 raft_consensus.cc:374] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:26.138983 26266 raft_consensus.cc:729] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc43bc44ee8e4347b1b0a7a98b92c38b, State: Initialized, Role: FOLLOWER
I20260923 15:10:26.139012 26267 raft_consensus.cc:729] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 697f08ef37504af7ad641f8761f0f46a, State: Initialized, Role: FOLLOWER
I20260923 15:10:26.139114 26267 consensus_queue.cc:260] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.139422 26267 ts_tablet_manager.cc:1428] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:26.139477 26121 heartbeater.cc:503] Master 127.7.250.126:40123 was elected leader, sending a full tablet report...
I20260923 15:10:26.139503 26268 tablet_bootstrap.cc:492] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: No bootstrap required, opened a new log
I20260923 15:10:26.139565 26268 ts_tablet_manager.cc:1397] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 15:10:26.141603 26268 raft_consensus.cc:348] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.141736 26268 raft_consensus.cc:374] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:26.141759 26268 raft_consensus.cc:729] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c900e41db5f485aa663a626ec274253, State: Initialized, Role: FOLLOWER
I20260923 15:10:26.141878 26268 consensus_queue.cc:260] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.142119 26268 ts_tablet_manager.cc:1428] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:26.142228 25990 heartbeater.cc:503] Master 127.7.250.126:40123 was elected leader, sending a full tablet report...
I20260923 15:10:26.142227 26266 consensus_queue.cc:260] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.142551 26266 ts_tablet_manager.cc:1428] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Time spent starting tablet: real 0.006s user 0.002s sys 0.002s
I20260923 15:10:26.142587 26252 heartbeater.cc:503] Master 127.7.250.126:40123 was elected leader, sending a full tablet report...
I20260923 15:10:26.158222 26273 raft_consensus.cc:482] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:26.158437 26273 raft_consensus.cc:504] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.158855 26273 leader_election.cc:288] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 697f08ef37504af7ad641f8761f0f46a (127.7.250.66:46041), bc43bc44ee8e4347b1b0a7a98b92c38b (127.7.250.67:41467)
I20260923 15:10:26.162398 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.159268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:57355 (local address 127.7.250.67:41467)
0923 15:10:26.159343 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.159346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.159478 (+ 132us) server_negotiation.cc:407] Connection header received
0923 15:10:26.159595 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.159596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.159616 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.159705 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:26.159893 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.160494 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.161606 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.161972 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.162075 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.162085 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.162281 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.162324 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 15:10:26.162482 26265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.159101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:43477 (local address 127.7.250.66:46041)
0923 15:10:26.159195 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.159198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.159210 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:10:26.159259 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.159260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.159275 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.159342 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:26.159522 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.160413 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.161613 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.162022 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.162126 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.162135 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.162361 (+ 226us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.162402 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 15:10:26.162505 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.159036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:46041 (local address 127.7.250.65:43477)
0923 15:10:26.159101 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.159116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:26.159177 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:26.159355 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:26.159356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:26.159358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:26.159442 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.159444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.160485 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.160486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.161524 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.161527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.162130 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.162131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.162211 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.162214 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:26.162338 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 15:10:26.162386 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":30}
I20260923 15:10:26.162415 26275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.159159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:41467 (local address 127.7.250.65:57355)
0923 15:10:26.159397 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.159412 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:26.159487 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:26.159732 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:26.159733 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:26.159735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:26.159809 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.159810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.160592 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.160593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.161513 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:26.161516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.162075 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.162076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:26.162152 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.162157 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:26.162275 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 15:10:26.162321 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":129,"threads_started":1}
I20260923 15:10:26.163007 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903a9701941948229c709aafcb49bab4" candidate_uuid: "5c900e41db5f485aa663a626ec274253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f08ef37504af7ad641f8761f0f46a" is_pre_election: true
I20260923 15:10:26.163173 26075 raft_consensus.cc:2454] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c900e41db5f485aa663a626ec274253 in term 0.
I20260923 15:10:26.163136 26206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903a9701941948229c709aafcb49bab4" candidate_uuid: "5c900e41db5f485aa663a626ec274253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" is_pre_election: true
I20260923 15:10:26.163309 26206 raft_consensus.cc:2454] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c900e41db5f485aa663a626ec274253 in term 0.
I20260923 15:10:26.163427 25878 leader_election.cc:302] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c900e41db5f485aa663a626ec274253, 697f08ef37504af7ad641f8761f0f46a; no voters:
I20260923 15:10:26.163606 26273 raft_consensus.cc:2781] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:10:26.163688 26273 raft_consensus.cc:482] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:26.163728 26273 raft_consensus.cc:3037] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:26.164517 26273 raft_consensus.cc:504] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.164672 26273 leader_election.cc:288] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [CANDIDATE]: Term 1 election: Requested vote from peers 697f08ef37504af7ad641f8761f0f46a (127.7.250.66:46041), bc43bc44ee8e4347b1b0a7a98b92c38b (127.7.250.67:41467)
I20260923 15:10:26.164919 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903a9701941948229c709aafcb49bab4" candidate_uuid: "5c900e41db5f485aa663a626ec274253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "697f08ef37504af7ad641f8761f0f46a"
I20260923 15:10:26.164923 26206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903a9701941948229c709aafcb49bab4" candidate_uuid: "5c900e41db5f485aa663a626ec274253" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b"
I20260923 15:10:26.165011 26206 raft_consensus.cc:3037] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:26.165014 26075 raft_consensus.cc:3037] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:26.165750 26206 raft_consensus.cc:2454] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c900e41db5f485aa663a626ec274253 in term 1.
I20260923 15:10:26.165920 26075 raft_consensus.cc:2454] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c900e41db5f485aa663a626ec274253 in term 1.
I20260923 15:10:26.165926 25878 leader_election.cc:302] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c900e41db5f485aa663a626ec274253, bc43bc44ee8e4347b1b0a7a98b92c38b; no voters:
I20260923 15:10:26.166144 26273 raft_consensus.cc:2781] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:26.166354 26273 raft_consensus.cc:686] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 1 LEADER]: Becoming Leader. State: Replica: 5c900e41db5f485aa663a626ec274253, State: Running, Role: LEADER
I20260923 15:10:26.166472 26273 consensus_queue.cc:237] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } }
I20260923 15:10:26.167447 25801 catalog_manager.cc:5495] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c900e41db5f485aa663a626ec274253 (127.7.250.65). New cstate: current_term: 1 leader_uuid: "5c900e41db5f485aa663a626ec274253" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c900e41db5f485aa663a626ec274253" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 35835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 } health_report { overall_health: UNKNOWN } } }
I20260923 15:10:26.174417 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.172049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.7.250.65:35835)
0923 15:10:26.172104 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:26.172106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:26.172114 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:10:26.172143 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:26.172144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:26.172162 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:26.172227 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:26.172350 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.172894 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.173698 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:26.173881 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:26.173937 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:26.174081 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:26.174174 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:26.174233 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:26.174325 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 15:10:26.174345 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:10:26.176259 26206 raft_consensus.cc:1261] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 1 FOLLOWER]: Refusing update from remote peer 5c900e41db5f485aa663a626ec274253: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:26.176285 26075 raft_consensus.cc:1261] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 1 FOLLOWER]: Refusing update from remote peer 5c900e41db5f485aa663a626ec274253: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:26.176641 26273 consensus_queue.cc:1035] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc43bc44ee8e4347b1b0a7a98b92c38b" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 41467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:26.176813 26276 consensus_queue.cc:1035] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [LEADER]: Connected to new peer: Peer: permanent_uuid: "697f08ef37504af7ad641f8761f0f46a" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 46041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:26.179263 26279 mvcc.cc:205] Tried to move back new op lower bound from 7332561822415163392 to 7332561822378541056. Current Snapshot: MvccSnapshot[applied={T|T < 7332561822415163392}]
I20260923 15:10:26.180070 26280 mvcc.cc:205] Tried to move back new op lower bound from 7332561822415163392 to 7332561822378541056. Current Snapshot: MvccSnapshot[applied={T|T < 7332561822415163392 or (T in {7332561822415163392})}]
I20260923 15:10:26.180260 26281 mvcc.cc:205] Tried to move back new op lower bound from 7332561822415163392 to 7332561822378541056. Current Snapshot: MvccSnapshot[applied={T|T < 7332561822415163392}]
I20260923 15:10:26.181623 25802 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:26.181707 25802 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47948:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:26.182566 25802 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd89a35ce978446790aaa01f6121bd4b: Sending DeleteTablet for 3 replicas of tablet 903a9701941948229c709aafcb49bab4
I20260923 15:10:26.182971 26055 tablet_service.cc:1514] Processing DeleteTablet for tablet 903a9701941948229c709aafcb49bab4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:26 UTC) from {username='slave'} at 127.0.0.1:60616
I20260923 15:10:26.182989 25924 tablet_service.cc:1514] Processing DeleteTablet for tablet 903a9701941948229c709aafcb49bab4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:26 UTC) from {username='slave'} at 127.0.0.1:46108
I20260923 15:10:26.183018 26186 tablet_service.cc:1514] Processing DeleteTablet for tablet 903a9701941948229c709aafcb49bab4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:26 UTC) from {username='slave'} at 127.0.0.1:50746
I20260923 15:10:26.183245 26289 tablet_replica.cc:330] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253: stopping tablet replica
I20260923 15:10:26.183405 26289 raft_consensus.cc:2229] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:10:26.183511 26289 raft_consensus.cc:2258] T 903a9701941948229c709aafcb49bab4 P 5c900e41db5f485aa663a626ec274253 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:26.183622 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25862
I20260923 15:10:26.184379 26290 tablet_replica.cc:330] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: stopping tablet replica
I20260923 15:10:26.184491 26290 raft_consensus.cc:2229] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:26.184610 26290 raft_consensus.cc:2258] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:26.184751 26291 tablet_replica.cc:330] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: stopping tablet replica
I20260923 15:10:26.184854 26291 raft_consensus.cc:2229] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:26.184952 26291 raft_consensus.cc:2258] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:26.185528 26291 ts_tablet_manager.cc:1905] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:26.185745 26290 ts_tablet_manager.cc:1905] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:26.187705 26291 ts_tablet_manager.cc:1918] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:26.187821 26291 log.cc:1196] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/903a9701941948229c709aafcb49bab4
I20260923 15:10:26.188094 26291 ts_tablet_manager.cc:1939] T 903a9701941948229c709aafcb49bab4 P bc43bc44ee8e4347b1b0a7a98b92c38b: Deleting consensus metadata
I20260923 15:10:26.188328 26290 ts_tablet_manager.cc:1918] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:26.188465 26290 log.cc:1196] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/903a9701941948229c709aafcb49bab4
I20260923 15:10:26.188499 25787 catalog_manager.cc:4841] TS bc43bc44ee8e4347b1b0a7a98b92c38b (127.7.250.67:41467): tablet 903a9701941948229c709aafcb49bab4 (table test-table [id=cc077e61c95e48b88d0dba637b1abc56]) successfully deleted
I20260923 15:10:26.188853 26290 ts_tablet_manager.cc:1939] T 903a9701941948229c709aafcb49bab4 P 697f08ef37504af7ad641f8761f0f46a: Deleting consensus metadata
I20260923 15:10:26.189409 25787 catalog_manager.cc:4841] TS 697f08ef37504af7ad641f8761f0f46a (127.7.250.66:46041): tablet 903a9701941948229c709aafcb49bab4 (table test-table [id=cc077e61c95e48b88d0dba637b1abc56]) successfully deleted
W20260923 15:10:26.190816 25789 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:35835 (error 108)
I20260923 15:10:26.191380 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25993
I20260923 15:10:26.191404 26264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:26.191236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:35835 (local address 127.0.0.1:46136)
0923 15:10:26.191309 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:26.191364 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:35835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260923 15:10:26.191992 25789 catalog_manager.cc:4584] TS 5c900e41db5f485aa663a626ec274253 (127.7.250.65:35835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 903a9701941948229c709aafcb49bab4: Network error: Client connection negotiation failed: client connection to 127.7.250.65:35835: connect: Connection refused (error 111)
I20260923 15:10:26.197602 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26124
I20260923 15:10:26.202716 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 25771
2026-09-23T15:10:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (701 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:26 dist-test-slave-bb3x krb5kdc[26295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:26 dist-test-slave-bb3x krb5kdc[26295](info): set up 2 sockets
Sep 23 15:10:26 dist-test-slave-bb3x krb5kdc[26295](info): commencing operation
krb5kdc: starting...
W20260923 15:10:27.319607 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:27.446362 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42179
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:42179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:27.527953 26308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:27.528165 26308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:27.528189 26308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:27.529278 26308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:27.529314 26308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:27.529328 26308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:27.529340 26308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:27.529353 26308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:27.530786 26308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:42179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:27.530979 26308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:27.531144 26308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:27.536880 26313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:27.536900 26308 server_base.cc:1034] running on GCE node
W20260923 15:10:27.536893 26316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:27.536880 26314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:27.537382 26308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:27.537693 26308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:27.538872 26308 hybrid_clock.cc:648] HybridClock initialized: now 1790176227538854 us; error 50 us; skew 500 ppm
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:27.541592 26308 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:27.542004 26308 webserver.cc:460] Webserver started at http://127.7.250.126:44525/ using document root <none> and password file <none>
I20260923 15:10:27.542201 26308 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:27.542304 26308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:27.542428 26308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:27.543469 26308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "57cabaac8dd846899ffce19cc13a77d5"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "763d1d701f8148141825e26183c40a99"
server_key_iv: "920ef92ab24bf32497b3bf2bf55270c5"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.543777 26308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "57cabaac8dd846899ffce19cc13a77d5"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "763d1d701f8148141825e26183c40a99"
server_key_iv: "920ef92ab24bf32497b3bf2bf55270c5"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.545054 26308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:10:27.545750 26323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.545953 26308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:27.546061 26308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "57cabaac8dd846899ffce19cc13a77d5"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "763d1d701f8148141825e26183c40a99"
server_key_iv: "920ef92ab24bf32497b3bf2bf55270c5"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.546139 26308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:27.556756 26308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:27.559007 26308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:27.559172 26308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:27.563055 26308 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:42179
I20260923 15:10:27.563102 26375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:42179 every 8 connection(s)
I20260923 15:10:27.563616 26308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:27.564332 26376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:27.566723 26376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5: Bootstrap starting.
I20260923 15:10:27.567337 26376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:27.567642 26376 log.cc:824] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:27.568364 26376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5: No bootstrap required, opened a new log
I20260923 15:10:27.569942 26376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cabaac8dd846899ffce19cc13a77d5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42179 } }
I20260923 15:10:27.570079 26376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:27.570111 26376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57cabaac8dd846899ffce19cc13a77d5, State: Initialized, Role: FOLLOWER
I20260923 15:10:27.570336 26376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cabaac8dd846899ffce19cc13a77d5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42179 } }
I20260923 15:10:27.570408 26376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:27.570430 26376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:27.570492 26376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:27.571420 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26308
I20260923 15:10:27.571561 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:27.571586 26376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cabaac8dd846899ffce19cc13a77d5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42179 } }
I20260923 15:10:27.571699 26376 leader_election.cc:302] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57cabaac8dd846899ffce19cc13a77d5; no voters:
I20260923 15:10:27.571817 8169 external_mini_cluster.cc:1428] Setting key 5c17375a35ab623e320fc84ba9ee20b3
I20260923 15:10:27.571884 26376 leader_election.cc:288] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:27.571916 26379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:27.572111 26379 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [term 1 LEADER]: Becoming Leader. State: Replica: 57cabaac8dd846899ffce19cc13a77d5, State: Running, Role: LEADER
I20260923 15:10:27.572157 26376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:27.572209 26379 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cabaac8dd846899ffce19cc13a77d5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42179 } }
I20260923 15:10:27.572532 26379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57cabaac8dd846899ffce19cc13a77d5. Latest consensus state: current_term: 1 leader_uuid: "57cabaac8dd846899ffce19cc13a77d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cabaac8dd846899ffce19cc13a77d5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42179 } } }
I20260923 15:10:27.572624 26379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:27.572706 26381 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57cabaac8dd846899ffce19cc13a77d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57cabaac8dd846899ffce19cc13a77d5" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42179 } } }
I20260923 15:10:27.572819 26381 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:27.573360 26387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:27.573938 26387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:27.575718 26387 catalog_manager.cc:1347] Generated new cluster ID: 244ebe2b6ff441f086adace933dc14b0
I20260923 15:10:27.575771 26387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:27.583992 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:27.572490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54780 (local address 127.7.250.126:42179)
0923 15:10:27.572747 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:27.572759 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:27.572798 (+ 39us) server_negotiation.cc:407] Connection header received
0923 15:10:27.572929 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:27.572938 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:27.573016 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:27.573157 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:27.574083 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.575403 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.576488 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.577087 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.577188 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:27.580032 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:27.580036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:27.580037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:27.580051 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:27.582195 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:27.582653 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:27.582656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:27.582657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:27.582784 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:27.582982 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:27.582983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:27.582984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:27.583334 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:27.583538 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:27.583613 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 15:10:27.583764 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":55,"threads_started":1}
I20260923 15:10:27.629964 26387 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:27.630751 26387 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:27.639873 26387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5: Generated new TSK 0
I20260923 15:10:27.640080 26387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:10:27.671322 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42179
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:27.755069 26402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:27.755299 26402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:27.755324 26402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:27.756527 26402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:27.756573 26402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:27.756600 26402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:27.758162 26402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:27.758433 26402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:27.758643 26402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:27.763681 26410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:27.763693 26408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:27.763701 26407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:27.764051 26402 server_base.cc:1034] running on GCE node
I20260923 15:10:27.764277 26402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:27.764571 26402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:27.765727 26402 hybrid_clock.cc:648] HybridClock initialized: now 1790176227765712 us; error 29 us; skew 500 ppm
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:27.768381 26402 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:10:27.768858 26402 webserver.cc:460] Webserver started at http://127.7.250.65:44301/ using document root <none> and password file <none>
I20260923 15:10:27.769109 26402 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:27.769156 26402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:27.769286 26402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:27.770418 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "0e20369ec0944e44b4a87360b00b678b"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "9407c4633841a1f88810ac6a47c5f59d"
server_key_iv: "622f10cc2b2a5ec713707fb964a53976"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.770790 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "0e20369ec0944e44b4a87360b00b678b"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "9407c4633841a1f88810ac6a47c5f59d"
server_key_iv: "622f10cc2b2a5ec713707fb964a53976"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.772269 26402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:10:27.773211 26417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.773450 26402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:27.773526 26402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "0e20369ec0944e44b4a87360b00b678b"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "9407c4633841a1f88810ac6a47c5f59d"
server_key_iv: "622f10cc2b2a5ec713707fb964a53976"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.773582 26402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:27.796679 26402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:27.799044 26402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:27.799216 26402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:27.799526 26402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:27.799973 26402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:27.800028 26402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.800057 26402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:27.800105 26402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.807003 26402 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:45345
I20260923 15:10:27.807052 26531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:45345 every 8 connection(s)
I20260923 15:10:27.807376 26402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:27.817333 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:27.808093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:57911 (local address 127.7.250.126:42179)
0923 15:10:27.808160 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:27.808162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:27.808373 (+ 211us) server_negotiation.cc:407] Connection header received
0923 15:10:27.808916 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:27.808918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:27.808935 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:27.808994 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:27.810092 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.810872 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.811961 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.812161 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.812231 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:27.814617 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:27.814618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:27.814619 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:27.814628 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:27.815905 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:27.816294 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:27.816295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:27.816295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:27.816344 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:27.816567 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:27.816567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:27.816567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:27.816729 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:27.816784 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:27.817130 (+ 346us) server_negotiation.cc:299] Negotiation successful
0923 15:10:27.817226 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:10:27.817312 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26402
I20260923 15:10:27.817425 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:10:27.817431 26534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:27.808066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42179 (local address 127.7.250.65:57911)
0923 15:10:27.808252 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:27.808290 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:27.808806 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:27.809067 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:27.809071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:27.809645 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:27.809908 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:27.809937 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.810974 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.810975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:27.811875 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:27.811878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.812273 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.812274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:27.812388 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:27.812964 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:27.812973 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:27.814525 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:27.815987 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:27.815993 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:27.815995 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:27.816226 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:27.816404 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:27.816405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:27.816405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:27.816512 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:27.816812 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:27.816819 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:27.816945 (+ 126us) client_negotiation.cc:769] Sending connection context
0923 15:10:27.817072 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:10:27.817222 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260923 15:10:27.817662 8169 external_mini_cluster.cc:1428] Setting key be2dee49126b8bd2a23a86406defdfb7
I20260923 15:10:27.818184 26532 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42179
I20260923 15:10:27.818318 26532 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:27.818506 26532 heartbeater.cc:511] Master 127.7.250.126:42179 requested a full tablet report, sending...
I20260923 15:10:27.819037 26340 ts_manager.cc:194] Registered new tserver with Master: 0e20369ec0944e44b4a87360b00b678b (127.7.250.65:45345)
I20260923 15:10:27.819690 26340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:57911
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:10:27.846940 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42179
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:27.929263 26537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:27.929494 26537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:27.929520 26537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:27.930859 26537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:27.930909 26537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:27.930938 26537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:27.932497 26537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:27.932719 26537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:27.932973 26537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:27.938355 26545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:27.938355 26543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:27.938515 26537 server_base.cc:1034] running on GCE node
W20260923 15:10:27.938304 26542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:27.938897 26537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:27.939179 26537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:27.940344 26537 hybrid_clock.cc:648] HybridClock initialized: now 1790176227940292 us; error 63 us; skew 500 ppm
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:27.943231 26537 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:10:27.943655 26537 webserver.cc:460] Webserver started at http://127.7.250.66:38373/ using document root <none> and password file <none>
I20260923 15:10:27.943862 26537 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:27.943903 26537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:27.943992 26537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:27.945097 26537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "84da6d90215a4e08bc955ed7650f102b"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "a4c87148711f245a69516127fa3142c8"
server_key_iv: "5c96319876c6c912796d6542e17c5bf5"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.945398 26537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "84da6d90215a4e08bc955ed7650f102b"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "a4c87148711f245a69516127fa3142c8"
server_key_iv: "5c96319876c6c912796d6542e17c5bf5"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.946782 26537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:27.947580 26552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.947786 26537 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:27.947872 26537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "84da6d90215a4e08bc955ed7650f102b"
format_stamp: "Formatted at 2026-09-23 15:10:27 on dist-test-slave-bb3x"
server_key: "a4c87148711f245a69516127fa3142c8"
server_key_iv: "5c96319876c6c912796d6542e17c5bf5"
server_key_version: "encryptionkey@0"
I20260923 15:10:27.947929 26537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:27.963672 26537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:27.966197 26537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:27.966391 26537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:27.966611 26537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:27.966987 26537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:27.967022 26537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.967069 26537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:27.967096 26537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:27.973309 26537 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:42677
I20260923 15:10:27.973359 26665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:42677 every 8 connection(s)
I20260923 15:10:27.973685 26537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:10:27 dist-test-slave-bb3x krb5kdc[26295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176227, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:27.982502 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26537
I20260923 15:10:27.982620 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:10:27.982859 8169 external_mini_cluster.cc:1428] Setting key 8ee25b625b350e70437b4b0dd01b68e2
I20260923 15:10:27.984560 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:27.974503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:47893 (local address 127.7.250.126:42179)
0923 15:10:27.974591 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:27.974593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:27.974859 (+ 266us) server_negotiation.cc:407] Connection header received
0923 15:10:27.975343 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:27.975344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:27.975363 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:27.975433 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:27.976431 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.976948 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.978061 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.978430 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.978616 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:27.981188 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:27.981190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:27.981190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:27.981198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:27.982908 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:27.983458 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:27.983459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:27.983460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:27.983531 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:27.983819 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:27.983820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:27.983821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:27.983953 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:27.984054 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:27.984363 (+ 309us) server_negotiation.cc:299] Negotiation successful
0923 15:10:27.984460 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:10:27.984839 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:27.974487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42179 (local address 127.7.250.66:47893)
0923 15:10:27.974711 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:27.974755 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:27.975252 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:27.975490 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:27.975494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:27.976074 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:27.976281 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:27.976287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.977058 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.977060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:27.977962 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:27.977965 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:27.978642 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:27.978643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:27.978802 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:27.979447 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:27.979457 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:27.981077 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:27.983000 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:27.983009 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:27.983012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:27.983359 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:27.983619 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:27.983620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:27.983620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:27.983744 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:27.984049 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:27.984057 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:27.984216 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 15:10:27.984339 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 15:10:27.984534 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":82,"threads_started":1}
I20260923 15:10:27.985368 26666 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42179
I20260923 15:10:27.985491 26666 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:27.985765 26666 heartbeater.cc:511] Master 127.7.250.126:42179 requested a full tablet report, sending...
I20260923 15:10:27.986203 26340 ts_manager.cc:194] Registered new tserver with Master: 84da6d90215a4e08bc955ed7650f102b (127.7.250.66:42677)
I20260923 15:10:27.986649 26340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:47893
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:10:28.014827 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42179
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:28.097327 26671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:28.097543 26671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:28.097568 26671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:28.098752 26671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:28.098796 26671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:28.098822 26671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:28.100240 26671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:42179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:28.100442 26671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:28.100632 26671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:28.106094 26676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.106094 26679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.106171 26671 server_base.cc:1034] running on GCE node
W20260923 15:10:28.106115 26677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.106616 26671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:28.106899 26671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:28.108103 26671 hybrid_clock.cc:648] HybridClock initialized: now 1790176228108101 us; error 49 us; skew 500 ppm
Sep 23 15:10:28 dist-test-slave-bb3x krb5kdc[26295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176228, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:28.110795 26671 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:10:28.111265 26671 webserver.cc:460] Webserver started at http://127.7.250.67:44023/ using document root <none> and password file <none>
I20260923 15:10:28.111466 26671 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:28.111506 26671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:28.111595 26671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:28.112618 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "e63fb9d275e545d09e33211e8e4a3be3"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "9598d4e4b54eb97b134daf6a752898c2"
server_key_iv: "be8b63f63becde909ea02d468d107613"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.112968 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "e63fb9d275e545d09e33211e8e4a3be3"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "9598d4e4b54eb97b134daf6a752898c2"
server_key_iv: "be8b63f63becde909ea02d468d107613"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.114367 26671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:10:28.115176 26686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.115348 26671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:10:28.115449 26671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "e63fb9d275e545d09e33211e8e4a3be3"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "9598d4e4b54eb97b134daf6a752898c2"
server_key_iv: "be8b63f63becde909ea02d468d107613"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.115531 26671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:28.152339 26671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:28.154790 26671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:28.154996 26671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:28.155265 26671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:28.155640 26671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:28.155704 26671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.155750 26671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:28.155779 26671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.161800 26671 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:33063
I20260923 15:10:28.161862 26799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:33063 every 8 connection(s)
I20260923 15:10:28.162271 26671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:10:28 dist-test-slave-bb3x krb5kdc[26295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176228, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:28.170619 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26671
I20260923 15:10:28.170727 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:10:28.170922 8169 external_mini_cluster.cc:1428] Setting key bfb2fece9f649351396785405f02b2e8
I20260923 15:10:28.173022 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.163106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:57175 (local address 127.7.250.126:42179)
0923 15:10:28.163199 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.163201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.163369 (+ 168us) server_negotiation.cc:407] Connection header received
0923 15:10:28.163991 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.163992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.164005 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.164093 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.165061 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.165573 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.166641 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.166837 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.166912 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.169060 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.169062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.169062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:28.169069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.171189 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:28.171788 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:28.171789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:28.171790 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:28.171863 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:28.172145 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:28.172145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:28.172146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:28.172268 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.172321 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.172866 (+ 545us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.172959 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 15:10:28.173089 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.163005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:42179 (local address 127.7.250.67:57175)
0923 15:10:28.163236 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.163284 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.163876 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.164184 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.164187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.164755 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:28.164955 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.164960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.165672 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.165673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.166544 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.166547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.166919 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.166921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.167044 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.167581 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:28.167590 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:28.168966 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:28.171280 (+ 2314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:28.171289 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:28.171291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:28.171706 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:28.171975 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:28.171976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:28.171976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:28.172096 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:28.172408 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:28.172415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:28.172573 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.172672 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.172891 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":109,"threads_started":1}
I20260923 15:10:28.173588 26800 heartbeater.cc:347] Connected to a master server at 127.7.250.126:42179
I20260923 15:10:28.173672 26800 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:28.173897 26800 heartbeater.cc:511] Master 127.7.250.126:42179 requested a full tablet report, sending...
I20260923 15:10:28.174403 26340 ts_manager.cc:194] Registered new tserver with Master: e63fb9d275e545d09e33211e8e4a3be3 (127.7.250.67:33063)
I20260923 15:10:28.174875 26340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:57175
I20260923 15:10:28.183218 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:10:28.191193 26384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.184734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.7.250.126:42179)
0923 15:10:28.184823 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.184825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.184838 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:10:28.184919 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.184919 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.184929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.184987 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.185559 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.186099 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.186950 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.187163 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.187222 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.188563 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.188564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.188564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:28.188571 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.189814 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:28.190206 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:28.190206 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:28.190207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:28.190323 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:28.190592 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:28.190593 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:28.190593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:28.190749 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.190839 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.191009 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.191064 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260923 15:10:28.193109 26340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:10:28.195293 26340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:10:28.201793 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.197764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40572 (local address 127.7.250.65:45345)
0923 15:10:28.198048 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.198053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.198066 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:10:28.198112 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.198115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.198134 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.198220 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.198560 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.199326 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.200715 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.201359 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.201446 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.201510 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.201660 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.201716 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":79,"threads_started":1}
I20260923 15:10:28.201783 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.197621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:45345 (local address 127.0.0.1:40572)
0923 15:10:28.197825 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.197868 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.197935 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.198385 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.198388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.198391 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.198471 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.198474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.199432 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.199433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.200608 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.200611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.201445 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.201446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.201526 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.201540 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.201640 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.201699 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":65,"threads_started":1}
I20260923 15:10:28.202157 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.197652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:42677 (local address 127.0.0.1:47108)
0923 15:10:28.197924 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.197932 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.197979 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.198448 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.198449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.198451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.198500 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.198502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.199864 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.199865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.200932 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.200935 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.201847 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.201848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.201930 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.201935 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.201967 (+ 32us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.202015 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":1,"thread_start_us":96,"threads_started":1}
I20260923 15:10:28.202272 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.198054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.7.250.67:33063)
0923 15:10:28.198249 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.198282 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.198293 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:10:28.198330 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.198332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.198347 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.198420 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.198610 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.199743 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.200889 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.201680 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.201755 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.201814 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.202061 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.202149 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260923 15:10:28.202608 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.197821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.7.250.66:42677)
0923 15:10:28.198249 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.198254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.198263 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:28.198324 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.198326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.198341 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.198576 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.198671 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.199754 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.201015 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.201749 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.202381 (+ 632us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.202444 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.202491 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.202532 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":91,"threads_started":1}
I20260923 15:10:28.202153 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.197855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33063 (local address 127.0.0.1:44684)
0923 15:10:28.198066 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.198074 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.198112 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.198500 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.198500 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.198502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.198536 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.198537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.199888 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.199889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.200788 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.200791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.201845 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.201847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.201930 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.201936 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.201984 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.202018 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":25,"thread_start_us":66,"threads_started":1}
I20260923 15:10:28.202986 26465 tablet_service.cc:1467] Processing CreateTablet for tablet 443653ebdae34ff4a1d53bf3599a3afb (DEFAULT_TABLE table=test-table [id=59a05787521740949098a6a4588f16be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:28.203385 26465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443653ebdae34ff4a1d53bf3599a3afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.203336 26734 tablet_service.cc:1467] Processing CreateTablet for tablet 443653ebdae34ff4a1d53bf3599a3afb (DEFAULT_TABLE table=test-table [id=59a05787521740949098a6a4588f16be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:28.203336 26600 tablet_service.cc:1467] Processing CreateTablet for tablet 443653ebdae34ff4a1d53bf3599a3afb (DEFAULT_TABLE table=test-table [id=59a05787521740949098a6a4588f16be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:28.203637 26600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443653ebdae34ff4a1d53bf3599a3afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.203637 26734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 443653ebdae34ff4a1d53bf3599a3afb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.206337 26817 tablet_bootstrap.cc:492] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b: Bootstrap starting.
I20260923 15:10:28.206943 26818 tablet_bootstrap.cc:492] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Bootstrap starting.
I20260923 15:10:28.207137 26817 tablet_bootstrap.cc:654] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.207633 26818 tablet_bootstrap.cc:654] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.207748 26817 log.cc:824] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.208144 26819 tablet_bootstrap.cc:492] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Bootstrap starting.
I20260923 15:10:28.208182 26818 log.cc:824] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.208600 26817 tablet_bootstrap.cc:492] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b: No bootstrap required, opened a new log
I20260923 15:10:28.208674 26817 ts_tablet_manager.cc:1397] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:28.208776 26819 tablet_bootstrap.cc:654] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.208953 26818 tablet_bootstrap.cc:492] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: No bootstrap required, opened a new log
I20260923 15:10:28.209035 26818 ts_tablet_manager.cc:1397] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:28.209332 26819 log.cc:824] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.210094 26819 tablet_bootstrap.cc:492] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: No bootstrap required, opened a new log
I20260923 15:10:28.210175 26819 ts_tablet_manager.cc:1397] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:10:28.210834 26818 raft_consensus.cc:348] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.210844 26817 raft_consensus.cc:348] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.210999 26818 raft_consensus.cc:374] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.211011 26817 raft_consensus.cc:374] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.211041 26818 raft_consensus.cc:729] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63fb9d275e545d09e33211e8e4a3be3, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.211045 26817 raft_consensus.cc:729] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e20369ec0944e44b4a87360b00b678b, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.211163 26818 consensus_queue.cc:260] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.211179 26817 consensus_queue.cc:260] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.211479 26817 ts_tablet_manager.cc:1428] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:28.211483 26818 ts_tablet_manager.cc:1428] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:10:28.211617 26532 heartbeater.cc:503] Master 127.7.250.126:42179 was elected leader, sending a full tablet report...
I20260923 15:10:28.211633 26800 heartbeater.cc:503] Master 127.7.250.126:42179 was elected leader, sending a full tablet report...
I20260923 15:10:28.212229 26819 raft_consensus.cc:348] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.212417 26819 raft_consensus.cc:374] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.212463 26819 raft_consensus.cc:729] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84da6d90215a4e08bc955ed7650f102b, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.212581 26819 consensus_queue.cc:260] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.212888 26819 ts_tablet_manager.cc:1428] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:10:28.212922 26666 heartbeater.cc:503] Master 127.7.250.126:42179 was elected leader, sending a full tablet report...
I20260923 15:10:28.216936 26825 raft_consensus.cc:482] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:28.217067 26825 raft_consensus.cc:504] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.217402 26825 leader_election.cc:288] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e20369ec0944e44b4a87360b00b678b (127.7.250.65:45345), e63fb9d275e545d09e33211e8e4a3be3 (127.7.250.67:33063)
I20260923 15:10:28.220619 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.217806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:60679 (local address 127.7.250.67:33063)
0923 15:10:28.217880 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.217882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.217952 (+ 70us) server_negotiation.cc:407] Connection header received
0923 15:10:28.218037 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.218038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.218047 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.218123 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.218291 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.218773 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.219858 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.220262 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.220346 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.220355 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.220513 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.220553 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 15:10:28.220633 26826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.217726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:33063 (local address 127.7.250.66:60679)
0923 15:10:28.217885 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.217895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.217967 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.218128 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.218129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.218131 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.218201 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.218207 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.218868 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.218869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.219765 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.219767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.220345 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.220346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.220422 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.220425 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.220504 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.220552 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":83,"threads_started":1}
I20260923 15:10:28.220695 26813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.217673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:35457 (local address 127.7.250.65:45345)
0923 15:10:28.217719 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.217722 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.217791 (+ 69us) server_negotiation.cc:407] Connection header received
0923 15:10:28.217896 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.217898 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.217909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.217978 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.218136 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.218714 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.219874 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.220290 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.220396 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.220405 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.220586 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.220626 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:10:28.220719 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.217606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:45345 (local address 127.7.250.66:35457)
0923 15:10:28.217727 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.217740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.217808 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.217983 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.217984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.217986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.218053 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.218055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.218818 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.218819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.219766 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.219768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.220396 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.220397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.220496 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.220499 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.220578 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.220621 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":1}
I20260923 15:10:28.220944 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443653ebdae34ff4a1d53bf3599a3afb" candidate_uuid: "84da6d90215a4e08bc955ed7650f102b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e20369ec0944e44b4a87360b00b678b" is_pre_election: true
I20260923 15:10:28.220942 26754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443653ebdae34ff4a1d53bf3599a3afb" candidate_uuid: "84da6d90215a4e08bc955ed7650f102b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63fb9d275e545d09e33211e8e4a3be3" is_pre_election: true
I20260923 15:10:28.221062 26485 raft_consensus.cc:2454] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84da6d90215a4e08bc955ed7650f102b in term 0.
I20260923 15:10:28.221104 26754 raft_consensus.cc:2454] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84da6d90215a4e08bc955ed7650f102b in term 0.
I20260923 15:10:28.221261 26554 leader_election.cc:302] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e20369ec0944e44b4a87360b00b678b, 84da6d90215a4e08bc955ed7650f102b; no voters:
I20260923 15:10:28.221436 26825 raft_consensus.cc:2781] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:10:28.221532 26825 raft_consensus.cc:482] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:28.221572 26825 raft_consensus.cc:3037] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:28.222401 26825 raft_consensus.cc:504] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.222553 26825 leader_election.cc:288] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [CANDIDATE]: Term 1 election: Requested vote from peers 0e20369ec0944e44b4a87360b00b678b (127.7.250.65:45345), e63fb9d275e545d09e33211e8e4a3be3 (127.7.250.67:33063)
I20260923 15:10:28.222790 26485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443653ebdae34ff4a1d53bf3599a3afb" candidate_uuid: "84da6d90215a4e08bc955ed7650f102b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e20369ec0944e44b4a87360b00b678b"
I20260923 15:10:28.222810 26754 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "443653ebdae34ff4a1d53bf3599a3afb" candidate_uuid: "84da6d90215a4e08bc955ed7650f102b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63fb9d275e545d09e33211e8e4a3be3"
I20260923 15:10:28.222887 26754 raft_consensus.cc:3037] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:28.222894 26485 raft_consensus.cc:3037] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:28.223682 26754 raft_consensus.cc:2454] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84da6d90215a4e08bc955ed7650f102b in term 1.
I20260923 15:10:28.223690 26485 raft_consensus.cc:2454] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84da6d90215a4e08bc955ed7650f102b in term 1.
I20260923 15:10:28.223867 26554 leader_election.cc:302] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e20369ec0944e44b4a87360b00b678b, 84da6d90215a4e08bc955ed7650f102b; no voters:
I20260923 15:10:28.223981 26825 raft_consensus.cc:2781] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:28.224162 26825 raft_consensus.cc:686] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 1 LEADER]: Becoming Leader. State: Replica: 84da6d90215a4e08bc955ed7650f102b, State: Running, Role: LEADER
W20260923 15:10:28.224188 26667 tablet.cc:2369] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:10:28.224275 26825 consensus_queue.cc:237] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } }
I20260923 15:10:28.225258 26340 catalog_manager.cc:5495] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b reported cstate change: term changed from 0 to 1, leader changed from <none> to 84da6d90215a4e08bc955ed7650f102b (127.7.250.66). New cstate: current_term: 1 leader_uuid: "84da6d90215a4e08bc955ed7650f102b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84da6d90215a4e08bc955ed7650f102b" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 42677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 } health_report { overall_health: UNKNOWN } } }
I20260923 15:10:28.238662 26814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.236152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.7.250.66:42677)
0923 15:10:28.236228 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.236230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.236239 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:28.236279 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.236280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.236300 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.236386 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:28.236530 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.237026 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.237807 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.238025 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.238108 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.238251 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:28.238389 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:28.238504 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.238534 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.238546 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:10:28.240633 26485 raft_consensus.cc:1261] T 443653ebdae34ff4a1d53bf3599a3afb P 0e20369ec0944e44b4a87360b00b678b [term 1 FOLLOWER]: Refusing update from remote peer 84da6d90215a4e08bc955ed7650f102b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:28.240633 26754 raft_consensus.cc:1261] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 1 FOLLOWER]: Refusing update from remote peer 84da6d90215a4e08bc955ed7650f102b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:28.241019 26827 consensus_queue.cc:1035] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63fb9d275e545d09e33211e8e4a3be3" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 33063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:28.241164 26825 consensus_queue.cc:1035] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e20369ec0944e44b4a87360b00b678b" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 45345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:28.243707 26830 mvcc.cc:205] Tried to move back new op lower bound from 7332561830870134784 to 7332561830807363584. Current Snapshot: MvccSnapshot[applied={T|T < 7332561830870134784}]
I20260923 15:10:28.243996 26832 mvcc.cc:205] Tried to move back new op lower bound from 7332561830870134784 to 7332561830807363584. Current Snapshot: MvccSnapshot[applied={T|T < 7332561830870134784}]
I20260923 15:10:28.244688 26831 mvcc.cc:205] Tried to move back new op lower bound from 7332561830870134784 to 7332561830807363584. Current Snapshot: MvccSnapshot[applied={T|T < 7332561830870134784}]
I20260923 15:10:28.246320 26340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:28.246431 26340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:28.247237 26340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57cabaac8dd846899ffce19cc13a77d5: Sending DeleteTablet for 3 replicas of tablet 443653ebdae34ff4a1d53bf3599a3afb
I20260923 15:10:28.247668 26734 tablet_service.cc:1514] Processing DeleteTablet for tablet 443653ebdae34ff4a1d53bf3599a3afb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:28 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:44684
I20260923 15:10:28.247642 26600 tablet_service.cc:1514] Processing DeleteTablet for tablet 443653ebdae34ff4a1d53bf3599a3afb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:28 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:47108
I20260923 15:10:28.247944 26840 tablet_replica.cc:330] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: stopping tablet replica
I20260923 15:10:28.247936 26465 tablet_service.cc:1514] Processing DeleteTablet for tablet 443653ebdae34ff4a1d53bf3599a3afb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:28 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:40572
I20260923 15:10:28.248055 26840 raft_consensus.cc:2229] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:28.247982 26841 tablet_replica.cc:330] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: stopping tablet replica
I20260923 15:10:28.248167 26840 raft_consensus.cc:2258] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:28.248255 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26402
I20260923 15:10:28.248307 26841 raft_consensus.cc:2229] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:10:28.248410 26841 raft_consensus.cc:2258] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:28.251408 26841 ts_tablet_manager.cc:1905] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:28.253077 26840 ts_tablet_manager.cc:1905] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:28.253319 26841 ts_tablet_manager.cc:1918] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:28.253393 26841 log.cc:1196] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/443653ebdae34ff4a1d53bf3599a3afb
I20260923 15:10:28.253701 26841 ts_tablet_manager.cc:1939] T 443653ebdae34ff4a1d53bf3599a3afb P 84da6d90215a4e08bc955ed7650f102b: Deleting consensus metadata
I20260923 15:10:28.254169 26324 catalog_manager.cc:4841] TS 84da6d90215a4e08bc955ed7650f102b (127.7.250.66:42677): tablet 443653ebdae34ff4a1d53bf3599a3afb (table test-table [id=59a05787521740949098a6a4588f16be]) successfully deleted
I20260923 15:10:28.254995 26840 ts_tablet_manager.cc:1918] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:28.255064 26840 log.cc:1196] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/443653ebdae34ff4a1d53bf3599a3afb
I20260923 15:10:28.255383 26840 ts_tablet_manager.cc:1939] T 443653ebdae34ff4a1d53bf3599a3afb P e63fb9d275e545d09e33211e8e4a3be3: Deleting consensus metadata
I20260923 15:10:28.255856 26325 catalog_manager.cc:4841] TS e63fb9d275e545d09e33211e8e4a3be3 (127.7.250.67:33063): tablet 443653ebdae34ff4a1d53bf3599a3afb (table test-table [id=59a05787521740949098a6a4588f16be]) successfully deleted
W20260923 15:10:28.261049 26325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:45345 (error 108)
I20260923 15:10:28.261288 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26537
I20260923 15:10:28.261848 26815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.261710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:45345 (local address 127.0.0.1:40586)
0923 15:10:28.261769 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.261805 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:45345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 15:10:28.261969 26325 catalog_manager.cc:4584] TS 0e20369ec0944e44b4a87360b00b678b (127.7.250.65:45345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 443653ebdae34ff4a1d53bf3599a3afb: Network error: Client connection negotiation failed: client connection to 127.7.250.65:45345: connect: Connection refused (error 111)
I20260923 15:10:28.266394 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26671
I20260923 15:10:28.271227 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26308
2026-09-23T15:10:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2069 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T15:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:28Z Disabled control of system clock
I20260923 15:10:28.300583 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:36639
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:36639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:28.385469 26847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:28.385717 26847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:28.385741 26847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:28.386940 26847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:28.386976 26847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:28.386989 26847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:28.387001 26847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:28.387014 26847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:28.388331 26847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:36639
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:36639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:28.388540 26847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:28.389192 26847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:28.394049 26855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.394075 26853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.394078 26852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.394315 26847 server_base.cc:1034] running on GCE node
I20260923 15:10:28.394557 26847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:28.394866 26847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:28.396026 26847 hybrid_clock.cc:648] HybridClock initialized: now 1790176228396013 us; error 28 us; skew 500 ppm
I20260923 15:10:28.397325 26847 webserver.cc:460] Webserver started at http://127.7.250.126:36615/ using document root <none> and password file <none>
I20260923 15:10:28.397583 26847 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:28.397699 26847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:28.397861 26847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:28.399013 26847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "ae403f76461e4438b89d192d9460ed5c"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "db3fd850e23301b2b6f7e227e8a042e6"
server_key_iv: "51e61d68b16fb267c5c169efb1f072a9"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.399394 26847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "ae403f76461e4438b89d192d9460ed5c"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "db3fd850e23301b2b6f7e227e8a042e6"
server_key_iv: "51e61d68b16fb267c5c169efb1f072a9"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.400810 26847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:10:28.401582 26861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.401750 26847 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:10:28.401851 26847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "ae403f76461e4438b89d192d9460ed5c"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "db3fd850e23301b2b6f7e227e8a042e6"
server_key_iv: "51e61d68b16fb267c5c169efb1f072a9"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.401937 26847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:28.415994 26847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:28.416349 26847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:28.416512 26847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:28.420751 26847 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:36639
I20260923 15:10:28.420812 26913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:36639 every 8 connection(s)
I20260923 15:10:28.421178 26847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:28.422209 26914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.424657 26914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c: Bootstrap starting.
I20260923 15:10:28.425439 26914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.425477 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26847
I20260923 15:10:28.425601 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:28.425843 8169 external_mini_cluster.cc:1428] Setting key f115f27ac8192b989cddc80dc28a68cc
I20260923 15:10:28.425827 26914 log.cc:824] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.426728 26914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c: No bootstrap required, opened a new log
I20260923 15:10:28.428963 26914 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae403f76461e4438b89d192d9460ed5c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36639 } }
I20260923 15:10:28.429167 26914 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.429209 26914 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae403f76461e4438b89d192d9460ed5c, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.429365 26914 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae403f76461e4438b89d192d9460ed5c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36639 } }
I20260923 15:10:28.429450 26914 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:28.429479 26914 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:28.429514 26914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:28.430157 26914 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae403f76461e4438b89d192d9460ed5c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36639 } }
I20260923 15:10:28.430305 26914 leader_election.cc:302] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae403f76461e4438b89d192d9460ed5c; no voters:
I20260923 15:10:28.430471 26914 leader_election.cc:288] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:28.430579 26919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:28.430760 26919 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [term 1 LEADER]: Becoming Leader. State: Replica: ae403f76461e4438b89d192d9460ed5c, State: Running, Role: LEADER
I20260923 15:10:28.430780 26914 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:28.430853 26919 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae403f76461e4438b89d192d9460ed5c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36639 } }
I20260923 15:10:28.431181 26919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae403f76461e4438b89d192d9460ed5c. Latest consensus state: current_term: 1 leader_uuid: "ae403f76461e4438b89d192d9460ed5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae403f76461e4438b89d192d9460ed5c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36639 } } }
I20260923 15:10:28.431268 26919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:28.431195 26920 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae403f76461e4438b89d192d9460ed5c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae403f76461e4438b89d192d9460ed5c" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 36639 } } }
I20260923 15:10:28.431471 26920 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:28.431650 26926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:28.432421 26926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:28.433195 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.426407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.7.250.126:36639)
0923 15:10:28.426625 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.426634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.426669 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:10:28.427250 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.427258 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.427336 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.427491 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.427729 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.429700 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.430702 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.431357 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.431483 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.432448 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.432451 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.432452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:28.432671 (+ 219us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.432713 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:28.432727 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.432794 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.432907 (+ 113us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.432984 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":81,"threads_started":1}
I20260923 15:10:28.434293 26926 catalog_manager.cc:1347] Generated new cluster ID: b630b9f28bf54ee694adcff947c5ac04
I20260923 15:10:28.434346 26926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:28.444015 26926 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:28.444527 26926 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:28.451253 26926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c: Generated new TSK 0
I20260923 15:10:28.451427 26926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:28.454793 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36639
--builtin_ntp_servers=127.7.250.84:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:28.535928 26938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:28.536144 26938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:28.536166 26938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:28.537359 26938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:28.537400 26938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:28.537426 26938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:28.538847 26938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:28.539114 26938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:28.539427 26938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:28.545507 26944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.545491 26946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.545492 26943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.545776 26938 server_base.cc:1034] running on GCE node
I20260923 15:10:28.545980 26938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:28.546286 26938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:28.547446 26938 hybrid_clock.cc:648] HybridClock initialized: now 1790176228547432 us; error 30 us; skew 500 ppm
I20260923 15:10:28.548866 26938 webserver.cc:460] Webserver started at http://127.7.250.65:36607/ using document root <none> and password file <none>
I20260923 15:10:28.549216 26938 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:28.549309 26938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:28.549484 26938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:28.550911 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "7af747e2a4b34f008f0c7ed28aa533af"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "7344a42942587c58ac855aff7063a8a0"
server_key_iv: "c71dabb06d10cb1c6fdc1fef58dec955"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.551321 26938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "7af747e2a4b34f008f0c7ed28aa533af"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "7344a42942587c58ac855aff7063a8a0"
server_key_iv: "c71dabb06d10cb1c6fdc1fef58dec955"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.552803 26938 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:28.553630 26952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.553838 26938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:28.553948 26938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "7af747e2a4b34f008f0c7ed28aa533af"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "7344a42942587c58ac855aff7063a8a0"
server_key_iv: "c71dabb06d10cb1c6fdc1fef58dec955"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.554042 26938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:28.566200 26938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:28.566568 26938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:28.566744 26938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:28.566993 26938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:28.567339 26938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:28.567394 26938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.567431 26938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:28.567480 26938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.573186 26938 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:38633
I20260923 15:10:28.573261 27065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:38633 every 8 connection(s)
I20260923 15:10:28.573611 26938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:10:28.578962 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.574421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:59403 (local address 127.7.250.126:36639)
0923 15:10:28.574474 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.574476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.574756 (+ 280us) server_negotiation.cc:407] Connection header received
0923 15:10:28.575265 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.575266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.575278 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.575329 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.575990 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.576502 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.577508 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.577761 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.577827 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.578701 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.578702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.578703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:28.578707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.578723 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:28.578727 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.578781 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.578874 (+ 93us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.578899 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:10:28.579109 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.574401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36639 (local address 127.7.250.65:59403)
0923 15:10:28.574593 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.574628 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.575174 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.575384 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.575388 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.575397 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:28.575905 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.575909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.576588 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.576589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.577403 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.577405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.577847 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.577847 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.577956 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.578559 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:28.578559 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:28.578588 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:28.578612 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:28.578783 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:28.578786 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:28.578788 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.578837 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.578917 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260923 15:10:28.579646 27066 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36639
I20260923 15:10:28.579757 27066 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:28.579860 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 26938
I20260923 15:10:28.579965 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:10:28.579977 27066 heartbeater.cc:511] Master 127.7.250.126:36639 requested a full tablet report, sending...
I20260923 15:10:28.580197 8169 external_mini_cluster.cc:1428] Setting key 596e8e036872567286af70d55a49828a
I20260923 15:10:28.580591 26878 ts_manager.cc:194] Registered new tserver with Master: 7af747e2a4b34f008f0c7ed28aa533af (127.7.250.65:38633)
I20260923 15:10:28.581372 26878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.65:59403
I20260923 15:10:28.581497 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36639
--builtin_ntp_servers=127.7.250.84:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:28.663980 27069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:28.664219 27069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:28.664255 27069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:28.665423 27069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:28.665488 27069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:28.665531 27069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:28.667457 27069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:28.667734 27069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:28.668032 27069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:28.673379 27075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.673379 27074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.673558 27069 server_base.cc:1034] running on GCE node
W20260923 15:10:28.673651 27077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.673887 27069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:28.674135 27069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:28.675315 27069 hybrid_clock.cc:648] HybridClock initialized: now 1790176228675306 us; error 45 us; skew 500 ppm
I20260923 15:10:28.676795 27069 webserver.cc:460] Webserver started at http://127.7.250.66:33625/ using document root <none> and password file <none>
I20260923 15:10:28.677048 27069 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:28.677126 27069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:28.677275 27069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:28.678453 27069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "6bf88157d961686113ab78811cfe614d"
server_key_iv: "8a721fff7eadbd8c0f4c94d00709d4f4"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.678825 27069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "6bf88157d961686113ab78811cfe614d"
server_key_iv: "8a721fff7eadbd8c0f4c94d00709d4f4"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.680213 27069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:28.681149 27083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.681373 27069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:28.681488 27069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "6bf88157d961686113ab78811cfe614d"
server_key_iv: "8a721fff7eadbd8c0f4c94d00709d4f4"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.681574 27069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:28.705168 27069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:28.705524 27069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:28.705701 27069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:28.705955 27069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:28.706372 27069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:28.706427 27069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.706468 27069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:28.706496 27069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.712339 27069 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:41301
I20260923 15:10:28.712385 27196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:41301 every 8 connection(s)
I20260923 15:10:28.712725 27069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:10:28.716997 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27069
I20260923 15:10:28.717136 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:10:28.717381 8169 external_mini_cluster.cc:1428] Setting key 41d2ab7df34b424b398152ab36d44b67
I20260923 15:10:28.718400 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36639
--builtin_ntp_servers=127.7.250.84:38651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:10:28.718459 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.713459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:55717 (local address 127.7.250.126:36639)
0923 15:10:28.713519 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.713521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.713794 (+ 273us) server_negotiation.cc:407] Connection header received
0923 15:10:28.714329 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.714330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.714342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.714397 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.715071 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.715558 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.716672 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.716993 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.717071 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.718141 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.718142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.718143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:28.718148 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.718166 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:28.718176 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.718241 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.718350 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.718379 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:10:28.718703 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.713447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36639 (local address 127.7.250.66:55717)
0923 15:10:28.713664 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.713701 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.714191 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.714462 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.714465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.714474 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:28.714964 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.714968 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.715728 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.715729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.716562 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.716565 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.717151 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.717153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.717319 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.717959 (+ 640us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:28.717960 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:28.717992 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:28.718027 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:28.718241 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:28.718248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:28.718273 (+ 25us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.718349 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.718467 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":94,"threads_started":1}
I20260923 15:10:28.719163 27197 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36639
I20260923 15:10:28.719274 27197 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:28.719480 27197 heartbeater.cc:511] Master 127.7.250.126:36639 requested a full tablet report, sending...
I20260923 15:10:28.719888 26878 ts_manager.cc:194] Registered new tserver with Master: 0f5c8b81dcd94eec9a08cd1ad8567f6c (127.7.250.66:41301)
I20260923 15:10:28.720445 26878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.66:55717
W20260923 15:10:28.801275 27200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:28.801524 27200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:28.801558 27200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:28.802762 27200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:28.802834 27200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:28.802876 27200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:28.804988 27200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:36639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:28.805272 27200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:28.805666 27200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:28.811127 27205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.811177 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:28.811182 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:28.811267 27200 server_base.cc:1034] running on GCE node
I20260923 15:10:28.811546 27200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:28.811796 27200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:28.812973 27200 hybrid_clock.cc:648] HybridClock initialized: now 1790176228812966 us; error 35 us; skew 500 ppm
I20260923 15:10:28.814405 27200 webserver.cc:460] Webserver started at http://127.7.250.67:36775/ using document root <none> and password file <none>
I20260923 15:10:28.814716 27200 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:28.814791 27200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:28.814946 27200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:28.816016 27200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "5715f5953ddc4495be4004a1917c17bc"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "45a35b831c44bd6ff00db5734999ef96"
server_key_iv: "9335c928046e80d6843733c9799714b7"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.816377 27200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "5715f5953ddc4495be4004a1917c17bc"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "45a35b831c44bd6ff00db5734999ef96"
server_key_iv: "9335c928046e80d6843733c9799714b7"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.817792 27200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:28.818660 27214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.818851 27200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:28.818948 27200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "5715f5953ddc4495be4004a1917c17bc"
format_stamp: "Formatted at 2026-09-23 15:10:28 on dist-test-slave-bb3x"
server_key: "45a35b831c44bd6ff00db5734999ef96"
server_key_iv: "9335c928046e80d6843733c9799714b7"
server_key_version: "encryptionkey@0"
I20260923 15:10:28.819031 27200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:28.842686 27200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:28.843034 27200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:28.843194 27200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:28.843438 27200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:28.843781 27200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:28.843834 27200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.843876 27200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:28.843919 27200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:28.850108 27200 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:36205
I20260923 15:10:28.850232 27327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:36205 every 8 connection(s)
I20260923 15:10:28.850564 27200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:10:28.854460 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27200
I20260923 15:10:28.854598 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:10:28.854817 8169 external_mini_cluster.cc:1428] Setting key 6f8971a9366e9745da279f5963b3c5bc
I20260923 15:10:28.856097 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.851304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:51349 (local address 127.7.250.126:36639)
0923 15:10:28.851374 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.851376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.851670 (+ 294us) server_negotiation.cc:407] Connection header received
0923 15:10:28.852084 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.852086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.852102 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.852162 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.852806 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.853284 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.854479 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.854761 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.854834 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.855749 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.855751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.855751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:28.855757 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.855775 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:28.855780 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.855863 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.855982 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.856014 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:10:28.856356 27330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.851288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:36639 (local address 127.7.250.67:51349)
0923 15:10:28.851495 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.851536 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.852004 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.852213 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.852217 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.852225 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:28.852733 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.852737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.853439 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.853441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.854385 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.854388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.854856 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.854857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.855005 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.855557 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:28.855558 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:28.855588 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:28.855613 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:28.855868 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:28.855874 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:28.855881 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.855960 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.856071 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260923 15:10:28.856858 27328 heartbeater.cc:347] Connected to a master server at 127.7.250.126:36639
I20260923 15:10:28.856953 27328 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:28.857101 27328 heartbeater.cc:511] Master 127.7.250.126:36639 requested a full tablet report, sending...
I20260923 15:10:28.857506 26878 ts_manager.cc:194] Registered new tserver with Master: 5715f5953ddc4495be4004a1917c17bc (127.7.250.67:36205)
I20260923 15:10:28.857836 26878 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.67:51349
I20260923 15:10:28.867012 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:10:28.871896 26918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.868436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.7.250.126:36639)
0923 15:10:28.868480 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.868482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.868592 (+ 110us) server_negotiation.cc:407] Connection header received
0923 15:10:28.868673 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.868674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.868683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.868733 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:28.869128 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.869649 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.870517 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.870737 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.870827 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.871583 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:28.871584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:28.871585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:28.871590 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:28.871604 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:28.871608 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:28.871728 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.871801 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.871812 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:10:28.873951 26878 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:10:28.875608 26878 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:10:28.882050 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.878055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:38633 (local address 127.0.0.1:47382)
0923 15:10:28.878476 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.878485 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.878547 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.878798 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.878800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.878802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.878848 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.878850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.879712 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.879713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.880947 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.880949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.881727 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.881728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.881806 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.881821 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.881890 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.881943 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":73,"threads_started":1}
I20260923 15:10:28.882088 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.878265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.7.250.65:38633)
0923 15:10:28.878455 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.878460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.878536 (+ 76us) server_negotiation.cc:407] Connection header received
0923 15:10:28.878619 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.878625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.878657 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.878732 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.878935 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.879622 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.881043 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.881640 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.881707 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.881774 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.881920 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.881979 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260923 15:10:28.882169 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.877829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41301 (local address 127.0.0.1:57762)
0923 15:10:28.878074 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.878117 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.878181 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.878772 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.878772 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.878775 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.878807 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.878809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.880150 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.880152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.881143 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.881146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.881905 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.881906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.881970 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.881975 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.882040 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.882084 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":89,"threads_started":1}
I20260923 15:10:28.882462 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.877989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57762 (local address 127.7.250.66:41301)
0923 15:10:28.878343 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.878349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.878361 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:10:28.878406 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.878409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.878427 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.878886 (+ 459us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.878962 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.880028 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.881232 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.881803 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.881877 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.881964 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.882288 (+ 324us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.882343 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":96,"threads_started":1}
I20260923 15:10:28.883085 27000 tablet_service.cc:1467] Processing CreateTablet for tablet a830e2a5eea842379eff4e5c69216445 (DEFAULT_TABLE table=test-table [id=7a666dff92674a71b3bf7f1ac99dd787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:28.883172 27131 tablet_service.cc:1467] Processing CreateTablet for tablet a830e2a5eea842379eff4e5c69216445 (DEFAULT_TABLE table=test-table [id=7a666dff92674a71b3bf7f1ac99dd787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:28.883495 27000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a830e2a5eea842379eff4e5c69216445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.883546 27131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a830e2a5eea842379eff4e5c69216445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.886744 27342 tablet_bootstrap.cc:492] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: Bootstrap starting.
I20260923 15:10:28.886862 27343 tablet_bootstrap.cc:492] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Bootstrap starting.
I20260923 15:10:28.887506 27343 tablet_bootstrap.cc:654] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.887545 27342 tablet_bootstrap.cc:654] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.887830 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.878102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.7.250.67:36205)
0923 15:10:28.878410 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:28.878413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:28.878423 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:10:28.878462 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:28.878463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:28.878478 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:28.878795 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:28.878862 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.880018 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.882003 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.887384 (+ 5381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.887473 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.887579 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:28.887684 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 15:10:28.887734 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":86,"threads_started":1}
I20260923 15:10:28.887835 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:28.877874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:36205 (local address 127.0.0.1:43466)
0923 15:10:28.878191 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:28.878203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:28.878283 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:28.878617 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:28.878621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:28.878624 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:28.878705 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.878707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.880137 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.880138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.881155 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:28.881156 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:28.887498 (+ 6342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:28.887500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:28.887577 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:28.887582 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:28.887667 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:10:28.887729 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":142,"threads_started":1}
I20260923 15:10:28.887862 27343 log.cc:824] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.888317 27342 log.cc:824] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.888654 27343 tablet_bootstrap.cc:492] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: No bootstrap required, opened a new log
I20260923 15:10:28.888738 27343 ts_tablet_manager.cc:1397] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:10:28.888662 27262 tablet_service.cc:1467] Processing CreateTablet for tablet a830e2a5eea842379eff4e5c69216445 (DEFAULT_TABLE table=test-table [id=7a666dff92674a71b3bf7f1ac99dd787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:28.889045 27262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a830e2a5eea842379eff4e5c69216445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:28.889088 27342 tablet_bootstrap.cc:492] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: No bootstrap required, opened a new log
I20260923 15:10:28.889176 27342 ts_tablet_manager.cc:1397] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:10:28.891402 27342 raft_consensus.cc:348] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:28.891510 27343 raft_consensus.cc:348] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:28.891574 27342 raft_consensus.cc:374] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.891680 27343 raft_consensus.cc:374] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.891709 27342 raft_consensus.cc:729] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af747e2a4b34f008f0c7ed28aa533af, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.891721 27343 raft_consensus.cc:729] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f5c8b81dcd94eec9a08cd1ad8567f6c, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.891841 27343 consensus_queue.cc:260] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:28.891842 27342 consensus_queue.cc:260] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:28.892154 27342 ts_tablet_manager.cc:1428] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:28.892174 27066 heartbeater.cc:503] Master 127.7.250.126:36639 was elected leader, sending a full tablet report...
I20260923 15:10:28.892194 27343 ts_tablet_manager.cc:1428] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260923 15:10:28.892333 27197 heartbeater.cc:503] Master 127.7.250.126:36639 was elected leader, sending a full tablet report...
I20260923 15:10:28.892943 27348 tablet_bootstrap.cc:492] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Bootstrap starting.
I20260923 15:10:28.893692 27348 tablet_bootstrap.cc:654] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:28.893993 27348 log.cc:824] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:28.894874 27348 tablet_bootstrap.cc:492] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: No bootstrap required, opened a new log
I20260923 15:10:28.894958 27348 ts_tablet_manager.cc:1397] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:10:28.896811 27348 raft_consensus.cc:348] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:28.896965 27348 raft_consensus.cc:374] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:28.897011 27348 raft_consensus.cc:729] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5715f5953ddc4495be4004a1917c17bc, State: Initialized, Role: FOLLOWER
I20260923 15:10:28.897145 27348 consensus_queue.cc:260] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:28.897428 27348 ts_tablet_manager.cc:1428] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260923 15:10:28.897461 27328 heartbeater.cc:503] Master 127.7.250.126:36639 was elected leader, sending a full tablet report...
W20260923 15:10:28.963322 27198 tablet.cc:2369] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:10:29.031373 27346 raft_consensus.cc:482] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:29.031600 27346 raft_consensus.cc:504] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:29.032023 27346 leader_election.cc:288] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f5c8b81dcd94eec9a08cd1ad8567f6c (127.7.250.66:41301), 5715f5953ddc4495be4004a1917c17bc (127.7.250.67:36205)
I20260923 15:10:29.035169 27340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:29.032315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:58831 (local address 127.7.250.67:36205)
0923 15:10:29.032379 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:29.032382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:29.032567 (+ 185us) server_negotiation.cc:407] Connection header received
0923 15:10:29.032656 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:29.032657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:29.032666 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:29.032743 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:29.032905 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.033365 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.034409 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.034782 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.034866 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:29.034875 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:29.035053 (+ 178us) server_negotiation.cc:299] Negotiation successful
0923 15:10:29.035096 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:10:29.035179 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:29.032251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:36205 (local address 127.7.250.65:58831)
0923 15:10:29.032489 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:29.032499 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:29.032596 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:29.032745 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:29.032746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:29.032747 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:29.032828 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:29.032830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.033447 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.033449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:29.034308 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:29.034310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.034877 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.034878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:29.034955 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:29.034958 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:29.035045 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 15:10:29.035086 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":98,"threads_started":1}
I20260923 15:10:29.035259 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:29.032311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:57577 (local address 127.7.250.66:41301)
0923 15:10:29.032361 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:29.032363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:29.032372 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:29.032418 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:29.032419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:29.032430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:29.032490 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:29.032784 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.033313 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.034433 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.034848 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.034942 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:29.034951 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:29.035135 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 15:10:29.035177 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 15:10:29.035274 27068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:29.032232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:41301 (local address 127.7.250.65:57577)
0923 15:10:29.032291 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:29.032302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:29.032345 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:29.032598 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:29.032600 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:29.032603 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:29.032684 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:29.032686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.033408 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.033409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:29.034329 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:29.034331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.034957 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.034958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:29.035053 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:29.035056 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:29.035129 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:10:29.035177 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":25}
I20260923 15:10:29.035720 27282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a830e2a5eea842379eff4e5c69216445" candidate_uuid: "7af747e2a4b34f008f0c7ed28aa533af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5715f5953ddc4495be4004a1917c17bc" is_pre_election: true
I20260923 15:10:29.035718 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a830e2a5eea842379eff4e5c69216445" candidate_uuid: "7af747e2a4b34f008f0c7ed28aa533af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" is_pre_election: true
I20260923 15:10:29.035888 27282 raft_consensus.cc:2454] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af747e2a4b34f008f0c7ed28aa533af in term 0.
I20260923 15:10:29.035899 27151 raft_consensus.cc:2454] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7af747e2a4b34f008f0c7ed28aa533af in term 0.
I20260923 15:10:29.036136 26956 leader_election.cc:302] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5c8b81dcd94eec9a08cd1ad8567f6c, 7af747e2a4b34f008f0c7ed28aa533af; no voters:
I20260923 15:10:29.036273 27346 raft_consensus.cc:2781] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:10:29.036331 27346 raft_consensus.cc:482] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:29.036355 27346 raft_consensus.cc:3037] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:29.037052 27346 raft_consensus.cc:504] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:29.037204 27346 leader_election.cc:288] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [CANDIDATE]: Term 1 election: Requested vote from peers 0f5c8b81dcd94eec9a08cd1ad8567f6c (127.7.250.66:41301), 5715f5953ddc4495be4004a1917c17bc (127.7.250.67:36205)
I20260923 15:10:29.037328 27151 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a830e2a5eea842379eff4e5c69216445" candidate_uuid: "7af747e2a4b34f008f0c7ed28aa533af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c"
I20260923 15:10:29.037418 27282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a830e2a5eea842379eff4e5c69216445" candidate_uuid: "7af747e2a4b34f008f0c7ed28aa533af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5715f5953ddc4495be4004a1917c17bc"
I20260923 15:10:29.037429 27151 raft_consensus.cc:3037] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:29.037487 27282 raft_consensus.cc:3037] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:29.038390 27282 raft_consensus.cc:2454] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af747e2a4b34f008f0c7ed28aa533af in term 1.
I20260923 15:10:29.038390 27151 raft_consensus.cc:2454] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7af747e2a4b34f008f0c7ed28aa533af in term 1.
I20260923 15:10:29.038581 26956 leader_election.cc:302] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f5c8b81dcd94eec9a08cd1ad8567f6c, 7af747e2a4b34f008f0c7ed28aa533af; no voters:
I20260923 15:10:29.038671 27346 raft_consensus.cc:2781] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:29.038823 27346 raft_consensus.cc:686] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 1 LEADER]: Becoming Leader. State: Replica: 7af747e2a4b34f008f0c7ed28aa533af, State: Running, Role: LEADER
I20260923 15:10:29.038919 27346 consensus_queue.cc:237] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } }
I20260923 15:10:29.039834 26877 catalog_manager.cc:5495] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af reported cstate change: term changed from 0 to 1, leader changed from <none> to 7af747e2a4b34f008f0c7ed28aa533af (127.7.250.65). New cstate: current_term: 1 leader_uuid: "7af747e2a4b34f008f0c7ed28aa533af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7af747e2a4b34f008f0c7ed28aa533af" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 38633 } health_report { overall_health: HEALTHY } } }
I20260923 15:10:29.065279 27341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:29.062823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47388 (local address 127.7.250.65:38633)
0923 15:10:29.062876 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:29.062888 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:29.062899 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:10:29.062949 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:29.062951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:29.062972 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:29.063055 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:29.063184 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.063698 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.064480 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:29.064684 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:29.064758 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:29.064907 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:29.065006 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:29.065132 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:29.065162 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 15:10:29.065175 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:10:29.067468 27282 raft_consensus.cc:1261] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 1 FOLLOWER]: Refusing update from remote peer 7af747e2a4b34f008f0c7ed28aa533af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:29.067469 27151 raft_consensus.cc:1261] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 1 FOLLOWER]: Refusing update from remote peer 7af747e2a4b34f008f0c7ed28aa533af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:29.067876 27352 consensus_queue.cc:1035] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f5c8b81dcd94eec9a08cd1ad8567f6c" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 41301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:29.068027 27346 consensus_queue.cc:1035] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [LEADER]: Connected to new peer: Peer: permanent_uuid: "5715f5953ddc4495be4004a1917c17bc" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:29.073225 26877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:29.073325 26877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:29.074141 26877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ae403f76461e4438b89d192d9460ed5c: Sending DeleteTablet for 3 replicas of tablet a830e2a5eea842379eff4e5c69216445
I20260923 15:10:29.074692 27131 tablet_service.cc:1514] Processing DeleteTablet for tablet a830e2a5eea842379eff4e5c69216445 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:29 UTC) from {username='slave'} at 127.0.0.1:57762
I20260923 15:10:29.074725 27000 tablet_service.cc:1514] Processing DeleteTablet for tablet a830e2a5eea842379eff4e5c69216445 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:29 UTC) from {username='slave'} at 127.0.0.1:47382
I20260923 15:10:29.074973 27366 tablet_replica.cc:330] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: stopping tablet replica
I20260923 15:10:29.075088 27366 raft_consensus.cc:2229] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:29.075199 27366 raft_consensus.cc:2258] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:29.075350 27365 tablet_replica.cc:330] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af: stopping tablet replica
I20260923 15:10:29.075309 27262 tablet_service.cc:1514] Processing DeleteTablet for tablet a830e2a5eea842379eff4e5c69216445 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:29 UTC) from {username='slave'} at 127.0.0.1:43466
I20260923 15:10:29.075433 27365 raft_consensus.cc:2229] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:10:29.075546 27365 raft_consensus.cc:2258] T a830e2a5eea842379eff4e5c69216445 P 7af747e2a4b34f008f0c7ed28aa533af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:29.075773 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26938
I20260923 15:10:29.078053 27366 ts_tablet_manager.cc:1905] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:29.080037 27366 ts_tablet_manager.cc:1918] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:29.080143 27366 log.cc:1196] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/a830e2a5eea842379eff4e5c69216445
I20260923 15:10:29.080343 27367 tablet_replica.cc:330] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: stopping tablet replica
I20260923 15:10:29.080425 27366 ts_tablet_manager.cc:1939] T a830e2a5eea842379eff4e5c69216445 P 0f5c8b81dcd94eec9a08cd1ad8567f6c: Deleting consensus metadata
I20260923 15:10:29.080477 27367 raft_consensus.cc:2229] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:29.080592 27367 raft_consensus.cc:2258] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:29.080974 26865 catalog_manager.cc:4841] TS 0f5c8b81dcd94eec9a08cd1ad8567f6c (127.7.250.66:41301): tablet a830e2a5eea842379eff4e5c69216445 (table test-table [id=7a666dff92674a71b3bf7f1ac99dd787]) successfully deleted
I20260923 15:10:29.081049 27367 ts_tablet_manager.cc:1905] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:10:29.081400 26865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:38633 (error 108)
I20260923 15:10:29.081696 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27069
I20260923 15:10:29.082396 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:29.081778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:38633 (local address 127.0.0.1:47402)
0923 15:10:29.082310 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:29.082358 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:38633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510}
I20260923 15:10:29.083354 27367 ts_tablet_manager.cc:1918] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:29.083434 27367 log.cc:1196] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/a830e2a5eea842379eff4e5c69216445
W20260923 15:10:29.083503 26865 catalog_manager.cc:4584] TS 7af747e2a4b34f008f0c7ed28aa533af (127.7.250.65:38633): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a830e2a5eea842379eff4e5c69216445: Network error: Client connection negotiation failed: client connection to 127.7.250.65:38633: connect: Connection refused (error 111)
I20260923 15:10:29.083705 27367 ts_tablet_manager.cc:1939] T a830e2a5eea842379eff4e5c69216445 P 5715f5953ddc4495be4004a1917c17bc: Deleting consensus metadata
I20260923 15:10:29.084079 26862 catalog_manager.cc:4841] TS 5715f5953ddc4495be4004a1917c17bc (127.7.250.67:36205): tablet a830e2a5eea842379eff4e5c69216445 (table test-table [id=7a666dff92674a71b3bf7f1ac99dd787]) successfully deleted
I20260923 15:10:29.086936 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27200
I20260923 15:10:29.092653 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 26847
2026-09-23T15:10:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (821 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:29 dist-test-slave-bb3x krb5kdc[27371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:29 dist-test-slave-bb3x krb5kdc[27371](info): set up 2 sockets
Sep 23 15:10:29 dist-test-slave-bb3x krb5kdc[27371](info): commencing operation
krb5kdc: starting...
W20260923 15:10:30.139806 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:30.273890 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40999
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:40999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:30.355201 27384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:30.355449 27384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:30.355489 27384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:30.356671 27384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:30.356738 27384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:30.356765 27384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:30.356796 27384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:30.356829 27384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:30.358496 27384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:40999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:40999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:30.358781 27384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:30.359017 27384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:30.364315 27390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:30.364319 27392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:30.364336 27384 server_base.cc:1034] running on GCE node
W20260923 15:10:30.364319 27389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:30.364879 27384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:30.365219 27384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:30.366493 27384 hybrid_clock.cc:648] HybridClock initialized: now 1790176230366488 us; error 35 us; skew 500 ppm
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:30.369140 27384 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:30.369635 27384 webserver.cc:460] Webserver started at http://127.7.250.126:44823/ using document root <none> and password file <none>
I20260923 15:10:30.369894 27384 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:30.369975 27384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:30.370127 27384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:30.371275 27384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "275c8cb787d342d8a6527fc7da74f3ea"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "a32fc5ef4894781854e694806466866b"
server_key_iv: "d84aafe7223b6a071c8babac55cd832e"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.371649 27384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "275c8cb787d342d8a6527fc7da74f3ea"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "a32fc5ef4894781854e694806466866b"
server_key_iv: "d84aafe7223b6a071c8babac55cd832e"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.373128 27384 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:30.373978 27399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.374161 27384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:30.374282 27384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "275c8cb787d342d8a6527fc7da74f3ea"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "a32fc5ef4894781854e694806466866b"
server_key_iv: "d84aafe7223b6a071c8babac55cd832e"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.374377 27384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:30.387691 27384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:30.389657 27384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:30.389828 27384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:30.393841 27384 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:40999
I20260923 15:10:30.393877 27451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:40999 every 8 connection(s)
I20260923 15:10:30.394291 27384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:30.395300 27452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:30.397579 27452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: Bootstrap starting.
I20260923 15:10:30.398182 27452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:30.398517 27452 log.cc:824] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:30.399156 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27384
I20260923 15:10:30.399245 27452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: No bootstrap required, opened a new log
I20260923 15:10:30.399293 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:30.399482 8169 external_mini_cluster.cc:1428] Setting key 8905efc562be52327eccbeaa4e4cac41
I20260923 15:10:30.401582 27452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275c8cb787d342d8a6527fc7da74f3ea" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40999 } }
I20260923 15:10:30.401800 27452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:30.401839 27452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 275c8cb787d342d8a6527fc7da74f3ea, State: Initialized, Role: FOLLOWER
I20260923 15:10:30.402026 27452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275c8cb787d342d8a6527fc7da74f3ea" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40999 } }
I20260923 15:10:30.402122 27452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:30.402154 27452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:30.402221 27452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:30.403028 27452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275c8cb787d342d8a6527fc7da74f3ea" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40999 } }
I20260923 15:10:30.403137 27452 leader_election.cc:302] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 275c8cb787d342d8a6527fc7da74f3ea; no voters:
I20260923 15:10:30.403298 27452 leader_election.cc:288] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:30.403378 27457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:30.403543 27457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [term 1 LEADER]: Becoming Leader. State: Replica: 275c8cb787d342d8a6527fc7da74f3ea, State: Running, Role: LEADER
I20260923 15:10:30.403578 27452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:30.403632 27457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275c8cb787d342d8a6527fc7da74f3ea" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40999 } }
I20260923 15:10:30.403968 27457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 275c8cb787d342d8a6527fc7da74f3ea. Latest consensus state: current_term: 1 leader_uuid: "275c8cb787d342d8a6527fc7da74f3ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275c8cb787d342d8a6527fc7da74f3ea" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40999 } } }
I20260923 15:10:30.404048 27457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:30.404277 27458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "275c8cb787d342d8a6527fc7da74f3ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "275c8cb787d342d8a6527fc7da74f3ea" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 40999 } } }
I20260923 15:10:30.404345 27458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea [sys.catalog]: This master's current role is: LEADER
W20260923 15:10:30.405668 27472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:10:30.405975 27472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:30.406057 27473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:30.406309 27473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:30.407856 27473 catalog_manager.cc:1347] Generated new cluster ID: 5d1ed4d3dd704d77b13d16a262b76f5c
I20260923 15:10:30.407905 27473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:30.412197 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.400071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59466 (local address 127.7.250.126:40999)
0923 15:10:30.400260 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.400270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.400306 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:10:30.400774 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.400783 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.400849 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.401021 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:30.401660 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.403410 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.404595 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.405245 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.405356 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.408346 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:30.408350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:30.408351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:30.408362 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:30.410344 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.410778 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.410780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.410781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.410885 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.411101 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.411102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.411102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.411407 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:30.411547 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.411679 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.411815 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260923 15:10:30.427588 27473 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:30.428350 27473 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:30.436947 27473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: Generated new TSK 0
I20260923 15:10:30.437158 27473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.250.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
Entry for principal kudu/127.7.250.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab.
I20260923 15:10:30.469293 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40999
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:30.552985 27478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:30.553231 27478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:30.553256 27478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:30.554502 27478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:30.554544 27478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:30.554571 27478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:30.555985 27478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.65.keytab
--principal=kudu/127.7.250.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:30.556164 27478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:30.556336 27478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:30.561343 27486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:30.561359 27483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:30.561374 27484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:30.561654 27478 server_base.cc:1034] running on GCE node
I20260923 15:10:30.561911 27478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:30.562196 27478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:30.563393 27478 hybrid_clock.cc:648] HybridClock initialized: now 1790176230563367 us; error 41 us; skew 500 ppm
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:30.566035 27478 init.cc:377] Logged in from keytab as kudu/127.7.250.65@KRBTEST.COM (short username kudu)
I20260923 15:10:30.566519 27478 webserver.cc:460] Webserver started at http://127.7.250.65:43873/ using document root <none> and password file <none>
I20260923 15:10:30.566753 27478 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:30.566823 27478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:30.566980 27478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:30.568084 27478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "1d85710c9122421c854f50abeb63b13a"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "55ffd263dad7b167fa1c5bf14228bc60"
server_key_iv: "115ecf8223f764f9ec611773fdd08209"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.568449 27478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "1d85710c9122421c854f50abeb63b13a"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "55ffd263dad7b167fa1c5bf14228bc60"
server_key_iv: "115ecf8223f764f9ec611773fdd08209"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.569782 27478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:30.570591 27493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.570768 27478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:30.570866 27478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "1d85710c9122421c854f50abeb63b13a"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "55ffd263dad7b167fa1c5bf14228bc60"
server_key_iv: "115ecf8223f764f9ec611773fdd08209"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.570948 27478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:30.585410 27478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:30.588443 27478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:30.588604 27478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:30.588896 27478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:30.589445 27478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:30.589504 27478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.589543 27478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:30.589565 27478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.597162 27478 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:40177
I20260923 15:10:30.597244 27606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:40177 every 8 connection(s)
I20260923 15:10:30.597496 27478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:10:30.604441 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27478
I20260923 15:10:30.604578 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:10:30.604828 8169 external_mini_cluster.cc:1428] Setting key 7fd5f849f0fd9b4dd03671db6802964a
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.65@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:30.612246 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.598221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:36067 (local address 127.7.250.126:40999)
0923 15:10:30.598288 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.598290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.598526 (+ 236us) server_negotiation.cc:407] Connection header received
0923 15:10:30.599120 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.599121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.599134 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.599193 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:30.600712 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.601376 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.603117 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.603438 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.603521 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.608679 (+ 5158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:30.608681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:30.608682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:30.608691 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:30.610566 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.611095 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.611096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.611097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.611173 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.611459 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.611460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.611460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.611598 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:30.611654 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.612033 (+ 379us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.612133 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:10:30.612884 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.598184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40999 (local address 127.7.250.65:36067)
0923 15:10:30.598377 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:30.598427 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:30.599013 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:30.599310 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:30.599314 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:30.600276 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:30.600572 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.600580 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.601478 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.601488 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.603009 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.603012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.603623 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.603624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.603788 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.605123 (+ 1335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:30.605133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:30.608558 (+ 3425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:30.610667 (+ 2109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:30.610676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:30.610679 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:30.610998 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:30.611248 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:30.611249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:30.611250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:30.611385 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:30.611689 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:30.611697 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:30.611861 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 15:10:30.612381 (+ 520us) client_negotiation.cc:240] Negotiation successful
0923 15:10:30.612603 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260923 15:10:30.613412 27607 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40999
I20260923 15:10:30.613529 27607 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:30.613736 27607 heartbeater.cc:511] Master 127.7.250.126:40999 requested a full tablet report, sending...
I20260923 15:10:30.614409 27416 ts_manager.cc:194] Registered new tserver with Master: 1d85710c9122421c854f50abeb63b13a (127.7.250.65:40177)
WARNING: no policy specified for kudu/127.7.250.66@KRBTEST.COM; defaulting to no policy
I20260923 15:10:30.615357 27416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.65@KRBTEST.COM'} at 127.7.250.65:36067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
Entry for principal kudu/127.7.250.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab.
I20260923 15:10:30.634320 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40999
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:30.719858 27612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:30.720086 27612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:30.720114 27612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:30.721427 27612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:30.721474 27612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:30.721504 27612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:30.723194 27612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.66.keytab
--principal=kudu/127.7.250.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:30.723467 27612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:30.723733 27612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:30.729435 27617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:30.729470 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:30.729461 27618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:30.729614 27612 server_base.cc:1034] running on GCE node
I20260923 15:10:30.729830 27612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:30.730127 27612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:30.731292 27612 hybrid_clock.cc:648] HybridClock initialized: now 1790176230731291 us; error 47 us; skew 500 ppm
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:30.737881 27612 init.cc:377] Logged in from keytab as kudu/127.7.250.66@KRBTEST.COM (short username kudu)
I20260923 15:10:30.738471 27612 webserver.cc:460] Webserver started at http://127.7.250.66:35227/ using document root <none> and password file <none>
I20260923 15:10:30.738691 27612 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:30.738734 27612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:30.738824 27612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:30.739903 27612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "c04be6d5a3cb450c8318aceb4f715196"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "c2f70021f7eb81090d5347e95943a49b"
server_key_iv: "0c410d02c14c29791cb52674989f772c"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.740216 27612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "c04be6d5a3cb450c8318aceb4f715196"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "c2f70021f7eb81090d5347e95943a49b"
server_key_iv: "0c410d02c14c29791cb52674989f772c"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.741640 27612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:30.742475 27627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.742697 27612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:30.742810 27612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "c04be6d5a3cb450c8318aceb4f715196"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "c2f70021f7eb81090d5347e95943a49b"
server_key_iv: "0c410d02c14c29791cb52674989f772c"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.742889 27612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:30.762145 27612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:30.764564 27612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:30.764740 27612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:30.764993 27612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:30.765321 27612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:30.765375 27612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.765417 27612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:30.765446 27612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.771512 27612 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:33477
I20260923 15:10:30.771552 27740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:33477 every 8 connection(s)
I20260923 15:10:30.771871 27612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.66@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:30.779850 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27612
I20260923 15:10:30.779958 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:10:30.780210 8169 external_mini_cluster.cc:1428] Setting key e8dd2a0bddc1ab2327796dc373698eb1
I20260923 15:10:30.782490 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.772739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:32909 (local address 127.7.250.126:40999)
0923 15:10:30.772807 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.772809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.773049 (+ 240us) server_negotiation.cc:407] Connection header received
0923 15:10:30.773618 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.773620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.773632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.773697 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:30.774870 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.775441 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.776481 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.776725 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.776795 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.779223 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:30.779224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:30.779224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:30.779231 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:30.781011 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.781440 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.781441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.781442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.781507 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.781763 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.781764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.781764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.781891 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:30.781993 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.782313 (+ 320us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.782392 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:10:30.782716 27743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.772728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40999 (local address 127.7.250.66:32909)
0923 15:10:30.772900 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:30.772938 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:30.773435 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:30.773801 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:30.773805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:30.774455 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:30.774769 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.774778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.775523 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.775524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.776357 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.776359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.776809 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.776810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.776962 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.777443 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:30.777452 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:30.779124 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:30.781091 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:30.781100 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:30.781103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:30.781351 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:30.781581 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:30.781582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:30.781582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:30.781693 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:30.781985 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:30.781992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:30.782139 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 15:10:30.782266 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:10:30.782458 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":84,"threads_started":1}
I20260923 15:10:30.783247 27741 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40999
I20260923 15:10:30.783370 27741 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:30.783618 27741 heartbeater.cc:511] Master 127.7.250.126:40999 requested a full tablet report, sending...
I20260923 15:10:30.784009 27416 ts_manager.cc:194] Registered new tserver with Master: c04be6d5a3cb450c8318aceb4f715196 (127.7.250.66:33477)
I20260923 15:10:30.784592 27416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.66@KRBTEST.COM'} at 127.7.250.66:32909
WARNING: no policy specified for kudu/127.7.250.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
Entry for principal kudu/127.7.250.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab.
I20260923 15:10:30.809454 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40999
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:33189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:30.894115 27746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:30.894405 27746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:30.894450 27746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:30.895681 27746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:30.895750 27746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:30.895798 27746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:30.897330 27746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:33189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.67.keytab
--principal=kudu/127.7.250.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:40999
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:30.897627 27746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:30.897866 27746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:30.903481 27751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:30.903585 27746 server_base.cc:1034] running on GCE node
W20260923 15:10:30.903450 27752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:30.903462 27754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:30.903894 27746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:30.904101 27746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:30.905237 27746 hybrid_clock.cc:648] HybridClock initialized: now 1790176230905227 us; error 29 us; skew 500 ppm
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:30.907663 27746 init.cc:377] Logged in from keytab as kudu/127.7.250.67@KRBTEST.COM (short username kudu)
I20260923 15:10:30.908120 27746 webserver.cc:460] Webserver started at http://127.7.250.67:41571/ using document root <none> and password file <none>
I20260923 15:10:30.908308 27746 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:30.908349 27746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:30.908439 27746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:30.909942 27746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "e5bbd81d8c424053b98d9e4e88c98612"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "9a348b6f33312d71168453123446ab27"
server_key_iv: "df69190175f16f2ce4f29b8d2ef469d2"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.910310 27746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "e5bbd81d8c424053b98d9e4e88c98612"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "9a348b6f33312d71168453123446ab27"
server_key_iv: "df69190175f16f2ce4f29b8d2ef469d2"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.911638 27746 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:30.912349 27761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.912528 27746 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:10:30.912652 27746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "e5bbd81d8c424053b98d9e4e88c98612"
format_stamp: "Formatted at 2026-09-23 15:10:30 on dist-test-slave-bb3x"
server_key: "9a348b6f33312d71168453123446ab27"
server_key_iv: "df69190175f16f2ce4f29b8d2ef469d2"
server_key_version: "encryptionkey@0"
I20260923 15:10:30.912740 27746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:30.929893 27746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:30.932132 27746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:30.932302 27746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:30.932567 27746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:30.932893 27746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:30.932948 27746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.932993 27746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:30.933027 27746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:30.939563 27746 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:35359
I20260923 15:10:30.939595 27874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:35359 every 8 connection(s)
I20260923 15:10:30.939921 27746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:10:30.945199 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27746
I20260923 15:10:30.945315 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:10:30.945534 8169 external_mini_cluster.cc:1428] Setting key b01ea145191b075b3cae79381e6c810d
Sep 23 15:10:30 dist-test-slave-bb3x krb5kdc[27371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176230, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.67@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:30.949713 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.940667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:42629 (local address 127.7.250.126:40999)
0923 15:10:30.940745 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.940747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.940932 (+ 185us) server_negotiation.cc:407] Connection header received
0923 15:10:30.941405 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.941406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.941416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.941524 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:30.942526 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.943088 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.944062 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.944270 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.944332 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.947052 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:30.947053 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:30.947054 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:30.947061 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:30.948257 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.948676 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.948677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.948677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.948734 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.949012 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.949013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.949013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.949134 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:30.949236 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.949537 (+ 301us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.949618 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 15:10:30.949924 27877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.940635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:40999 (local address 127.7.250.67:42629)
0923 15:10:30.940796 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:30.940837 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:30.941323 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:30.941581 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:30.941585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:30.942145 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:30.942420 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.942426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.943176 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.943177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.943966 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.943968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.944347 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.944348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.944444 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.945000 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:30.945011 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:30.946951 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:30.948343 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:30.948350 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:30.948352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:30.948579 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:30.948811 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:30.948811 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:30.948812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:30.948926 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:30.949229 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:30.949236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:30.949401 (+ 165us) client_negotiation.cc:769] Sending connection context
0923 15:10:30.949513 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 15:10:30.949681 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260923 15:10:30.950379 27875 heartbeater.cc:347] Connected to a master server at 127.7.250.126:40999
I20260923 15:10:30.950491 27875 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:30.950695 27875 heartbeater.cc:511] Master 127.7.250.126:40999 requested a full tablet report, sending...
I20260923 15:10:30.951135 27416 ts_manager.cc:194] Registered new tserver with Master: e5bbd81d8c424053b98d9e4e88c98612 (127.7.250.67:35359)
I20260923 15:10:30.951514 27416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.250.67@KRBTEST.COM'} at 127.7.250.67:42629
I20260923 15:10:30.958184 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:10:30.966504 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.959770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59478 (local address 127.7.250.126:40999)
0923 15:10:30.959809 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.959811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.959954 (+ 143us) server_negotiation.cc:407] Connection header received
0923 15:10:30.959998 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.959998 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.960007 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.960059 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:30.960753 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.961287 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.962165 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.962424 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.962482 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.963823 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:30.963824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:30.963824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:30.963831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:30.965248 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.965681 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.965682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.965682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.965731 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:30.965913 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:30.965913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:30.965914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:30.966068 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:30.966146 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.966369 (+ 223us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.966433 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:10:30.968174 27416 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59478:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:10:30.969758 27416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:10:30.976965 27886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.972395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33477 (local address 127.0.0.1:33482)
0923 15:10:30.972556 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:30.972599 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:30.972667 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:30.973088 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:30.973091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:30.973095 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:30.973211 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.973214 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.974600 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.974602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.975703 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.975706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.976568 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.976694 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.976710 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:10:30.976776 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:10:30.976838 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":83,"thread_start_us":57,"threads_started":1}
I20260923 15:10:30.977028 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.972654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36082 (local address 127.7.250.65:40177)
0923 15:10:30.972841 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.972846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.972973 (+ 127us) server_negotiation.cc:407] Connection header received
0923 15:10:30.973212 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.973215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.973235 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.973310 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:30.973521 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.974470 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976093 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.976629 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976697 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.976752 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.976854 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.976894 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260923 15:10:30.976984 27887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.972401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:40177 (local address 127.0.0.1:36082)
0923 15:10:30.972907 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:30.972918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:30.972974 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:30.973392 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:30.973392 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:30.973394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:30.973446 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.973447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.974576 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.974577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.975989 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.975992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.976702 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.976770 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.976775 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:30.976838 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:10:30.976888 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":61,"thread_start_us":84,"threads_started":1}
I20260923 15:10:30.977344 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.972705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.7.250.66:33477)
0923 15:10:30.972887 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.972892 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.972904 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:10:30.972945 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.972948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.972964 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.973081 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:30.973287 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.974485 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.975797 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.976475 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976560 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.976626 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.977205 (+ 579us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.977259 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":107,"threads_started":1}
I20260923 15:10:30.977934 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.972407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:35359 (local address 127.0.0.1:48198)
0923 15:10:30.973104 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:30.973114 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:30.973159 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:30.973356 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:30.973357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:30.973358 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:30.973417 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.973419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.974685 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.974686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.975994 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:30.975996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.977010 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.977010 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:30.977747 (+ 737us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.977752 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:30.977829 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:10:30.977870 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"mutex_wait_us":78,"thread_start_us":100,"threads_started":1}
I20260923 15:10:30.977968 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:30.972662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48198 (local address 127.7.250.67:35359)
0923 15:10:30.972922 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:30.972926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:30.973218 (+ 292us) server_negotiation.cc:407] Connection header received
0923 15:10:30.973264 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:30.973266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:30.973283 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:30.973360 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:30.973481 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.974587 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976087 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:30.976869 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:30.976928 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:30.976977 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:30.977847 (+ 870us) server_negotiation.cc:299] Negotiation successful
0923 15:10:30.977890 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":78,"threads_started":1}
I20260923 15:10:30.977989 27541 tablet_service.cc:1467] Processing CreateTablet for tablet d2e77e3f0cfe47c48de1b8a1973d3bce (DEFAULT_TABLE table=test-table [id=3a14d0cfb18f4563a18a5f9b0b99c176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:30.977980 27675 tablet_service.cc:1467] Processing CreateTablet for tablet d2e77e3f0cfe47c48de1b8a1973d3bce (DEFAULT_TABLE table=test-table [id=3a14d0cfb18f4563a18a5f9b0b99c176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:30.978389 27675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2e77e3f0cfe47c48de1b8a1973d3bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:30.978471 27541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2e77e3f0cfe47c48de1b8a1973d3bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:30.981881 27892 tablet_bootstrap.cc:492] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a: Bootstrap starting.
I20260923 15:10:30.982420 27893 tablet_bootstrap.cc:492] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Bootstrap starting.
I20260923 15:10:30.982345 27809 tablet_service.cc:1467] Processing CreateTablet for tablet d2e77e3f0cfe47c48de1b8a1973d3bce (DEFAULT_TABLE table=test-table [id=3a14d0cfb18f4563a18a5f9b0b99c176]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:30.982685 27809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2e77e3f0cfe47c48de1b8a1973d3bce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:30.982719 27892 tablet_bootstrap.cc:654] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:30.983117 27893 tablet_bootstrap.cc:654] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:30.983304 27892 log.cc:824] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:30.983685 27893 log.cc:824] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:30.984196 27892 tablet_bootstrap.cc:492] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a: No bootstrap required, opened a new log
I20260923 15:10:30.984280 27892 ts_tablet_manager.cc:1397] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:10:30.984660 27893 tablet_bootstrap.cc:492] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: No bootstrap required, opened a new log
I20260923 15:10:30.984730 27893 ts_tablet_manager.cc:1397] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:10:30.985711 27896 tablet_bootstrap.cc:492] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Bootstrap starting.
I20260923 15:10:30.986398 27896 tablet_bootstrap.cc:654] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:30.986362 27892 raft_consensus.cc:348] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:30.986658 27892 raft_consensus.cc:374] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:30.986737 27892 raft_consensus.cc:729] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d85710c9122421c854f50abeb63b13a, State: Initialized, Role: FOLLOWER
I20260923 15:10:30.986865 27892 consensus_queue.cc:260] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:30.986902 27893 raft_consensus.cc:348] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:30.987016 27896 log.cc:824] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:30.987072 27893 raft_consensus.cc:374] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:30.987120 27893 raft_consensus.cc:729] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c04be6d5a3cb450c8318aceb4f715196, State: Initialized, Role: FOLLOWER
I20260923 15:10:30.987187 27892 ts_tablet_manager.cc:1428] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:10:30.987267 27893 consensus_queue.cc:260] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:30.987320 27607 heartbeater.cc:503] Master 127.7.250.126:40999 was elected leader, sending a full tablet report...
I20260923 15:10:30.987584 27893 ts_tablet_manager.cc:1428] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:30.987666 27741 heartbeater.cc:503] Master 127.7.250.126:40999 was elected leader, sending a full tablet report...
I20260923 15:10:30.987869 27896 tablet_bootstrap.cc:492] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: No bootstrap required, opened a new log
I20260923 15:10:30.987929 27896 ts_tablet_manager.cc:1397] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:10:30.989497 27896 raft_consensus.cc:348] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:30.989621 27896 raft_consensus.cc:374] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:30.989642 27896 raft_consensus.cc:729] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5bbd81d8c424053b98d9e4e88c98612, State: Initialized, Role: FOLLOWER
I20260923 15:10:30.989759 27896 consensus_queue.cc:260] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:30.990023 27896 ts_tablet_manager.cc:1428] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:10:30.990061 27875 heartbeater.cc:503] Master 127.7.250.126:40999 was elected leader, sending a full tablet report...
W20260923 15:10:31.022475 27742 tablet.cc:2369] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:10:31.040264 27897 raft_consensus.cc:482] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:31.040401 27897 raft_consensus.cc:504] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:31.040939 27897 leader_election.cc:288] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c04be6d5a3cb450c8318aceb4f715196 (127.7.250.66:33477), e5bbd81d8c424053b98d9e4e88c98612 (127.7.250.67:35359)
I20260923 15:10:31.044917 27890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.041369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:48653 (local address 127.7.250.67:35359)
0923 15:10:31.041439 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.041441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.041477 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:10:31.041528 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.041529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.041539 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.041631 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.041833 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.042566 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.043924 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.044515 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.044635 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.044643 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.044813 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.044854 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:10:31.044917 27891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.041286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:50005 (local address 127.7.250.66:33477)
0923 15:10:31.041348 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.041350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.041367 (+ 17us) server_negotiation.cc:407] Connection header received
0923 15:10:31.041412 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.041413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.041425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.041505 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.041714 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.042507 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.043924 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.044514 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.044598 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.044610 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.044802 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.044844 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:10:31.044929 27609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.041152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33477 (local address 127.7.250.65:50005)
0923 15:10:31.041207 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.041218 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.041266 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.041527 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.041528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.041530 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.041607 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.041609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.042582 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.042582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.043820 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.043823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.044621 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.044622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.044707 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.044712 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.044773 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.044826 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":7}
I20260923 15:10:31.044934 27901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.041163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:35359 (local address 127.7.250.65:48653)
0923 15:10:31.041403 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.041412 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.041455 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.041703 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.041704 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.041706 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.041765 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.041767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.042671 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.042672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.043820 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.043823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.044675 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.044676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.044745 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.044749 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.044800 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.044828 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":53,"thread_start_us":120,"threads_started":1}
I20260923 15:10:31.045277 27695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e77e3f0cfe47c48de1b8a1973d3bce" candidate_uuid: "1d85710c9122421c854f50abeb63b13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04be6d5a3cb450c8318aceb4f715196" is_pre_election: true
I20260923 15:10:31.045454 27695 raft_consensus.cc:2454] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d85710c9122421c854f50abeb63b13a in term 0.
I20260923 15:10:31.045593 27829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e77e3f0cfe47c48de1b8a1973d3bce" candidate_uuid: "1d85710c9122421c854f50abeb63b13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5bbd81d8c424053b98d9e4e88c98612" is_pre_election: true
I20260923 15:10:31.045717 27496 leader_election.cc:302] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d85710c9122421c854f50abeb63b13a, c04be6d5a3cb450c8318aceb4f715196; no voters:
I20260923 15:10:31.045763 27829 raft_consensus.cc:2454] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d85710c9122421c854f50abeb63b13a in term 0.
I20260923 15:10:31.045853 27897 raft_consensus.cc:2781] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:10:31.045915 27897 raft_consensus.cc:482] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:31.045938 27897 raft_consensus.cc:3037] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.046706 27897 raft_consensus.cc:504] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:31.046849 27897 leader_election.cc:288] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [CANDIDATE]: Term 1 election: Requested vote from peers c04be6d5a3cb450c8318aceb4f715196 (127.7.250.66:33477), e5bbd81d8c424053b98d9e4e88c98612 (127.7.250.67:35359)
I20260923 15:10:31.047009 27695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e77e3f0cfe47c48de1b8a1973d3bce" candidate_uuid: "1d85710c9122421c854f50abeb63b13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04be6d5a3cb450c8318aceb4f715196"
I20260923 15:10:31.047032 27829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d2e77e3f0cfe47c48de1b8a1973d3bce" candidate_uuid: "1d85710c9122421c854f50abeb63b13a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5bbd81d8c424053b98d9e4e88c98612"
I20260923 15:10:31.047091 27695 raft_consensus.cc:3037] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.047128 27829 raft_consensus.cc:3037] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.047839 27695 raft_consensus.cc:2454] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d85710c9122421c854f50abeb63b13a in term 1.
I20260923 15:10:31.047847 27829 raft_consensus.cc:2454] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d85710c9122421c854f50abeb63b13a in term 1.
I20260923 15:10:31.048020 27496 leader_election.cc:302] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d85710c9122421c854f50abeb63b13a, c04be6d5a3cb450c8318aceb4f715196; no voters:
I20260923 15:10:31.048125 27897 raft_consensus.cc:2781] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:31.048265 27897 raft_consensus.cc:686] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [term 1 LEADER]: Becoming Leader. State: Replica: 1d85710c9122421c854f50abeb63b13a, State: Running, Role: LEADER
I20260923 15:10:31.048377 27897 consensus_queue.cc:237] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } }
I20260923 15:10:31.049261 27416 catalog_manager.cc:5495] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d85710c9122421c854f50abeb63b13a (127.7.250.65). New cstate: current_term: 1 leader_uuid: "1d85710c9122421c854f50abeb63b13a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d85710c9122421c854f50abeb63b13a" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 40177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 } health_report { overall_health: UNKNOWN } } }
I20260923 15:10:31.068635 27889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.066128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36084 (local address 127.7.250.65:40177)
0923 15:10:31.066172 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.066174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.066182 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:10:31.066297 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.066299 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.066328 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.066408 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:31.066544 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.067047 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.067803 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.067998 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.068084 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.068263 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:31.068367 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:31.068456 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.068493 (+ 37us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.068514 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:10:31.070555 27695 raft_consensus.cc:1261] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 1 FOLLOWER]: Refusing update from remote peer 1d85710c9122421c854f50abeb63b13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:31.070588 27829 raft_consensus.cc:1261] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 1 FOLLOWER]: Refusing update from remote peer 1d85710c9122421c854f50abeb63b13a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:31.070911 27902 consensus_queue.cc:1035] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5bbd81d8c424053b98d9e4e88c98612" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:31.071069 27897 consensus_queue.cc:1035] T d2e77e3f0cfe47c48de1b8a1973d3bce P 1d85710c9122421c854f50abeb63b13a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c04be6d5a3cb450c8318aceb4f715196" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 33477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:31.073866 27906 mvcc.cc:205] Tried to move back new op lower bound from 7332561842461753344 to 7332561842374754304. Current Snapshot: MvccSnapshot[applied={T|T < 7332561842461753344}]
I20260923 15:10:31.076480 27414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:31.076591 27414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59478:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:31.077364 27414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 275c8cb787d342d8a6527fc7da74f3ea: Sending DeleteTablet for 3 replicas of tablet d2e77e3f0cfe47c48de1b8a1973d3bce
I20260923 15:10:31.077721 27541 tablet_service.cc:1514] Processing DeleteTablet for tablet d2e77e3f0cfe47c48de1b8a1973d3bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:31 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:36082
I20260923 15:10:31.077800 27675 tablet_service.cc:1514] Processing DeleteTablet for tablet d2e77e3f0cfe47c48de1b8a1973d3bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:31 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:33482
I20260923 15:10:31.077987 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27478
I20260923 15:10:31.078234 27809 tablet_service.cc:1514] Processing DeleteTablet for tablet d2e77e3f0cfe47c48de1b8a1973d3bce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:31 UTC) from {username='kudu', principal='kudu/127.7.250.126@KRBTEST.COM'} at 127.0.0.1:48198
I20260923 15:10:31.078845 27916 tablet_replica.cc:330] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: stopping tablet replica
I20260923 15:10:31.078923 27916 raft_consensus.cc:2229] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:31.078995 27916 raft_consensus.cc:2258] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:31.079391 27916 ts_tablet_manager.cc:1905] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:31.080798 27916 ts_tablet_manager.cc:1918] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:31.080852 27916 log.cc:1196] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/d2e77e3f0cfe47c48de1b8a1973d3bce
I20260923 15:10:31.081094 27916 ts_tablet_manager.cc:1939] T d2e77e3f0cfe47c48de1b8a1973d3bce P e5bbd81d8c424053b98d9e4e88c98612: Deleting consensus metadata
I20260923 15:10:31.081466 27403 catalog_manager.cc:4841] TS e5bbd81d8c424053b98d9e4e88c98612 (127.7.250.67:35359): tablet d2e77e3f0cfe47c48de1b8a1973d3bce (table test-table [id=3a14d0cfb18f4563a18a5f9b0b99c176]) successfully deleted
I20260923 15:10:31.082463 27915 tablet_replica.cc:330] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: stopping tablet replica
I20260923 15:10:31.082592 27915 raft_consensus.cc:2229] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:31.082726 27915 raft_consensus.cc:2258] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:31.083114 27915 ts_tablet_manager.cc:1905] T d2e77e3f0cfe47c48de1b8a1973d3bce P c04be6d5a3cb450c8318aceb4f715196: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:10:31.083910 27400 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:40177 (error 108)
I20260923 15:10:31.084172 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27612
I20260923 15:10:31.086943 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.086792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:40177 (local address 127.0.0.1:36086)
0923 15:10:31.086857 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.086906 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:40177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260923 15:10:31.087172 27400 catalog_manager.cc:4584] TS 1d85710c9122421c854f50abeb63b13a (127.7.250.65:40177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d2e77e3f0cfe47c48de1b8a1973d3bce: Network error: Client connection negotiation failed: client connection to 127.7.250.65:40177: connect: Connection refused (error 111)
I20260923 15:10:31.089076 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.088964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:33477 (local address 127.0.0.1:33498)
0923 15:10:31.089012 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.089045 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.66:33477: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 15:10:31.089156 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27746
I20260923 15:10:31.098980 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27384
2026-09-23T15:10:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2006 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T15:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:31Z Disabled control of system clock
I20260923 15:10:31.127350 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37505
--webserver_interface=127.7.250.126
--webserver_port=0
--builtin_ntp_servers=127.7.250.84:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:37505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:31.209055 27921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:31.209296 27921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:31.209319 27921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:31.210547 27921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:31.210594 27921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:31.210608 27921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:31.210620 27921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:31.210632 27921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:31.211970 27921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:37505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:37505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:31.212180 27921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:31.212419 27921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:31.217459 27929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:31.217442 27927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:31.217520 27926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:31.217694 27921 server_base.cc:1034] running on GCE node
I20260923 15:10:31.217967 27921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:31.218303 27921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:31.219520 27921 hybrid_clock.cc:648] HybridClock initialized: now 1790176231219494 us; error 39 us; skew 500 ppm
I20260923 15:10:31.221028 27921 webserver.cc:460] Webserver started at http://127.7.250.126:32959/ using document root <none> and password file <none>
I20260923 15:10:31.221316 27921 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:31.221390 27921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:31.221530 27921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:31.222774 27921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "eceb7004a8c54710bbb69de767f17bc7"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "ee1cead61f877992ec6657089e11fa13"
server_key_iv: "f86f77ed0ec64fd9feeb84f064ddf294"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.223153 27921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "eceb7004a8c54710bbb69de767f17bc7"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "ee1cead61f877992ec6657089e11fa13"
server_key_iv: "f86f77ed0ec64fd9feeb84f064ddf294"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.224601 27921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:31.225433 27935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.225698 27921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:31.225824 27921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "eceb7004a8c54710bbb69de767f17bc7"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "ee1cead61f877992ec6657089e11fa13"
server_key_iv: "f86f77ed0ec64fd9feeb84f064ddf294"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.225961 27921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:31.236336 27921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:31.236750 27921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:31.236917 27921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:31.241659 27921 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:37505
I20260923 15:10:31.241798 27987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:37505 every 8 connection(s)
I20260923 15:10:31.242102 27921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:31.243191 27988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:31.245473 27988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7: Bootstrap starting.
I20260923 15:10:31.246011 27988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:31.246285 27988 log.cc:824] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:31.247022 27988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7: No bootstrap required, opened a new log
I20260923 15:10:31.248626 27988 raft_consensus.cc:348] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eceb7004a8c54710bbb69de767f17bc7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37505 } }
I20260923 15:10:31.248773 27988 raft_consensus.cc:374] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:31.248795 27988 raft_consensus.cc:729] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eceb7004a8c54710bbb69de767f17bc7, State: Initialized, Role: FOLLOWER
I20260923 15:10:31.248932 27988 consensus_queue.cc:260] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eceb7004a8c54710bbb69de767f17bc7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37505 } }
I20260923 15:10:31.249013 27988 raft_consensus.cc:388] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:31.249037 27988 raft_consensus.cc:482] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:31.249070 27988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.249531 27988 raft_consensus.cc:504] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eceb7004a8c54710bbb69de767f17bc7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37505 } }
I20260923 15:10:31.249617 27988 leader_election.cc:302] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eceb7004a8c54710bbb69de767f17bc7; no voters:
I20260923 15:10:31.249740 27988 leader_election.cc:288] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:31.250123 27991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:31.250331 27991 raft_consensus.cc:686] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [term 1 LEADER]: Becoming Leader. State: Replica: eceb7004a8c54710bbb69de767f17bc7, State: Running, Role: LEADER
I20260923 15:10:31.250368 27988 sys_catalog.cc:564] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:31.250428 27991 consensus_queue.cc:237] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eceb7004a8c54710bbb69de767f17bc7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37505 } }
I20260923 15:10:31.250818 27992 sys_catalog.cc:455] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eceb7004a8c54710bbb69de767f17bc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eceb7004a8c54710bbb69de767f17bc7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37505 } } }
I20260923 15:10:31.250902 27992 sys_catalog.cc:458] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:31.251035 27993 sys_catalog.cc:455] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eceb7004a8c54710bbb69de767f17bc7. Latest consensus state: current_term: 1 leader_uuid: "eceb7004a8c54710bbb69de767f17bc7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eceb7004a8c54710bbb69de767f17bc7" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 37505 } } }
I20260923 15:10:31.251195 27993 sys_catalog.cc:458] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:31.251528 27998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:31.251823 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 27921
I20260923 15:10:31.251952 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:31.252125 27998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:31.252228 8169 external_mini_cluster.cc:1428] Setting key c436c0fc35ad53b8c64c7d22b43bd039
I20260923 15:10:31.254139 27998 catalog_manager.cc:1347] Generated new cluster ID: e18f24bac1094e77a233072259f38c2b
I20260923 15:10:31.254199 27998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:31.258874 28010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.252889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35262 (local address 127.7.250.126:37505)
0923 15:10:31.253209 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.253217 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.253245 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:10:31.253341 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.253347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.253402 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.253598 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:31.253854 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.255243 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.256451 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.257099 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.257229 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.257958 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:31.257961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:31.257962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:31.258194 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:31.258238 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:31.258299 (+ 61us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:31.258373 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.258456 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.258651 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":76,"threads_started":1}
I20260923 15:10:31.265182 27998 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:31.265929 27998 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:31.268781 27998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7: Generated new TSK 0
I20260923 15:10:31.268961 27998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:31.277076 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.65:0
--local_ip_for_outbound_sockets=127.7.250.65
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37505
--builtin_ntp_servers=127.7.250.84:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:10:31.373401 28012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:31.373647 28012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:31.373685 28012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:31.374910 28012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:31.374970 28012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:31.375015 28012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.65
I20260923 15:10:31.376420 28012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.250.65
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.65
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:31.376695 28012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:31.377002 28012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:31.383376 28017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:31.383421 28018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:31.383566 28012 server_base.cc:1034] running on GCE node
W20260923 15:10:31.383376 28020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:31.384037 28012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:31.384343 28012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:31.385545 28012 hybrid_clock.cc:648] HybridClock initialized: now 1790176231385497 us; error 55 us; skew 500 ppm
I20260923 15:10:31.387105 28012 webserver.cc:460] Webserver started at http://127.7.250.65:43353/ using document root <none> and password file <none>
I20260923 15:10:31.387390 28012 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:31.387462 28012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:31.387622 28012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:31.389181 28012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data/instance:
uuid: "183ca4e897fd457d9fe4cb4a71e072e8"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "69348e9b6f5367ce4639e27e4c44dd25"
server_key_iv: "0e09528ca188d33aec2c244b08930e6b"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.389650 28012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance:
uuid: "183ca4e897fd457d9fe4cb4a71e072e8"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "69348e9b6f5367ce4639e27e4c44dd25"
server_key_iv: "0e09528ca188d33aec2c244b08930e6b"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.391506 28012 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.001s
I20260923 15:10:31.392509 28026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.392751 28012 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:31.392855 28012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal
uuid: "183ca4e897fd457d9fe4cb4a71e072e8"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "69348e9b6f5367ce4639e27e4c44dd25"
server_key_iv: "0e09528ca188d33aec2c244b08930e6b"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.392940 28012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:31.422765 28012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:31.423143 28012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:31.423301 28012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:31.423580 28012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:31.423959 28012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:31.424002 28012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.424034 28012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:31.424062 28012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.431133 28012 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.65:43631
I20260923 15:10:31.431190 28141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.65:43631 every 8 connection(s)
I20260923 15:10:31.431617 28012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/data/info.pb
I20260923 15:10:31.433360 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28012
I20260923 15:10:31.433471 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-0/wal/instance
I20260923 15:10:31.433765 8169 external_mini_cluster.cc:1428] Setting key 431ea4b145794de46c13c854666ef70f
I20260923 15:10:31.434894 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.66:0
--local_ip_for_outbound_sockets=127.7.250.66
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37505
--builtin_ntp_servers=127.7.250.84:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:10:31.437458 28010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.432408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.65:54111 (local address 127.7.250.126:37505)
0923 15:10:31.432460 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.432462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.432742 (+ 280us) server_negotiation.cc:407] Connection header received
0923 15:10:31.433285 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.433286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.433300 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.433351 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:31.434243 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.434859 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.435935 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.436133 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.436218 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.437123 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:31.437124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:31.437125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:31.437130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:31.437146 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:31.437150 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:31.437201 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.437312 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.437343 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:10:31.437618 28144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.432417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37505 (local address 127.7.250.65:54111)
0923 15:10:31.432607 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.432647 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.433142 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.433420 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.433425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.433434 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:31.434133 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.434146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.434954 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.434955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.435830 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.435833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.436231 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.436232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.436373 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.436972 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:31.436973 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:31.437002 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:31.437025 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:31.437219 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:31.437225 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:31.437228 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.437300 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.437390 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260923 15:10:31.438159 28142 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37505
I20260923 15:10:31.438297 28142 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:31.438498 28142 heartbeater.cc:511] Master 127.7.250.126:37505 requested a full tablet report, sending...
I20260923 15:10:31.439035 27952 ts_manager.cc:194] Registered new tserver with Master: 183ca4e897fd457d9fe4cb4a71e072e8 (127.7.250.65:43631)
I20260923 15:10:31.439780 27952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.65:54111
W20260923 15:10:31.517295 28145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:31.517530 28145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:31.517566 28145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:31.518788 28145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:31.518854 28145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:31.518900 28145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.66
I20260923 15:10:31.520370 28145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.250.66
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.66
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:31.520632 28145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:31.520931 28145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:31.525962 28150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:31.526038 28153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:31.525978 28145 server_base.cc:1034] running on GCE node
W20260923 15:10:31.525962 28151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:31.526425 28145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:31.526718 28145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:31.527875 28145 hybrid_clock.cc:648] HybridClock initialized: now 1790176231527856 us; error 34 us; skew 500 ppm
I20260923 15:10:31.529121 28145 webserver.cc:460] Webserver started at http://127.7.250.66:36631/ using document root <none> and password file <none>
I20260923 15:10:31.529428 28145 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:31.529476 28145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:31.529673 28145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:31.530871 28145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data/instance:
uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "f97ddb30db3ddf1f089d780cf5cf6d3a"
server_key_iv: "3cb918ef9c6e68d7475e07f6afc2b4aa"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.531234 28145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance:
uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "f97ddb30db3ddf1f089d780cf5cf6d3a"
server_key_iv: "3cb918ef9c6e68d7475e07f6afc2b4aa"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.532548 28145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:31.533320 28159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.533507 28145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:31.533604 28145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal
uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "f97ddb30db3ddf1f089d780cf5cf6d3a"
server_key_iv: "3cb918ef9c6e68d7475e07f6afc2b4aa"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.533694 28145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:31.560463 28145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:31.560808 28145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:31.560967 28145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:31.561211 28145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:31.561568 28145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:31.561622 28145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.561657 28145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:31.561684 28145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.567634 28145 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.66:43471
I20260923 15:10:31.567688 28273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.66:43471 every 8 connection(s)
I20260923 15:10:31.568009 28145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/data/info.pb
I20260923 15:10:31.570640 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28145
I20260923 15:10:31.570755 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal/instance
I20260923 15:10:31.570959 8169 external_mini_cluster.cc:1428] Setting key d357f11af117f53522b75226dfe54710
I20260923 15:10:31.571996 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.250.67:0
--local_ip_for_outbound_sockets=127.7.250.67
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37505
--builtin_ntp_servers=127.7.250.84:42601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:10:31.574316 28010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.568813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:56361 (local address 127.7.250.126:37505)
0923 15:10:31.568889 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.568891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.569162 (+ 271us) server_negotiation.cc:407] Connection header received
0923 15:10:31.569640 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.569641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.569650 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.569711 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:31.570491 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.571234 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.572592 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.572802 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.572871 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.573990 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:31.573992 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:31.573992 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:31.573998 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:31.574018 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:31.574023 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:31.574090 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.574171 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.574208 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:10:31.574512 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.568811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37505 (local address 127.7.250.66:56361)
0923 15:10:31.569004 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.569074 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.569556 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.569777 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.569780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.569789 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:31.570402 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.570408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.571345 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.571348 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.572489 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.572493 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.572898 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.572899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.573065 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.573825 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:31.573826 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:31.573854 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:31.573877 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:31.574095 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:31.574099 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:31.574101 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.574158 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.574241 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260923 15:10:31.575052 28274 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37505
I20260923 15:10:31.575134 28274 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:31.575343 28274 heartbeater.cc:511] Master 127.7.250.126:37505 requested a full tablet report, sending...
I20260923 15:10:31.575743 27952 ts_manager.cc:194] Registered new tserver with Master: 7d5f20ee780d4a4a91ab0f99f74ce9b4 (127.7.250.66:43471)
I20260923 15:10:31.576336 27952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.66:56361
W20260923 15:10:31.659886 28277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:31.660091 28277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:31.660113 28277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:31.661237 28277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:31.661274 28277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:31.661299 28277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.250.67
I20260923 15:10:31.662696 28277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.250.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.250.67
--webserver_port=0
--tserver_master_addrs=127.7.250.126:37505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.250.67
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:31.662886 28277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:31.663133 28277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:31.668268 28283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:31.668260 28285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:31.668262 28282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:31.668606 28277 server_base.cc:1034] running on GCE node
I20260923 15:10:31.668869 28277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:31.669121 28277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:31.670325 28277 hybrid_clock.cc:648] HybridClock initialized: now 1790176231670309 us; error 33 us; skew 500 ppm
I20260923 15:10:31.671674 28277 webserver.cc:460] Webserver started at http://127.7.250.67:45321/ using document root <none> and password file <none>
I20260923 15:10:31.671993 28277 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:31.672070 28277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:31.672214 28277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:31.673273 28277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data/instance:
uuid: "6d8ecd60897a49588d379b3e125dacc0"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "48ebba143fa3fef7fa364483ed3a847f"
server_key_iv: "05ef28858be1e1ecfeb06906042c80e9"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.673642 28277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance:
uuid: "6d8ecd60897a49588d379b3e125dacc0"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "48ebba143fa3fef7fa364483ed3a847f"
server_key_iv: "05ef28858be1e1ecfeb06906042c80e9"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.675100 28277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:10:31.675936 28291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.676137 28277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:31.676241 28277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal
uuid: "6d8ecd60897a49588d379b3e125dacc0"
format_stamp: "Formatted at 2026-09-23 15:10:31 on dist-test-slave-bb3x"
server_key: "48ebba143fa3fef7fa364483ed3a847f"
server_key_iv: "05ef28858be1e1ecfeb06906042c80e9"
server_key_version: "encryptionkey@0"
I20260923 15:10:31.676328 28277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:31.697810 28277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:31.698135 28277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:31.698323 28277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:31.698623 28277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:10:31.698971 28277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:10:31.699025 28277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.699062 28277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:10:31.699088 28277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:31.704605 28277 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.67:35185
I20260923 15:10:31.704703 28405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.67:35185 every 8 connection(s)
I20260923 15:10:31.705025 28277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/data/info.pb
I20260923 15:10:31.708065 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28277
I20260923 15:10:31.708180 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal/instance
I20260923 15:10:31.708388 8169 external_mini_cluster.cc:1428] Setting key 62c1903e1589d4ddd01c6ea9c710ae55
I20260923 15:10:31.711016 28010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.705800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.67:34429 (local address 127.7.250.126:37505)
0923 15:10:31.705880 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.705882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.706095 (+ 213us) server_negotiation.cc:407] Connection header received
0923 15:10:31.706690 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.706692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.706703 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.706762 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:31.707426 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.708058 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.709572 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.709772 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.709829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.710784 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:31.710785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:31.710785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:31.710790 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:31.710804 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:31.710808 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:31.710864 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.710929 (+ 65us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.710956 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:10:31.711253 28408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.705756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:37505 (local address 127.7.250.67:34429)
0923 15:10:31.705962 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.705994 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.706591 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.706818 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.706821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.706829 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:31.707334 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.707339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.708184 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.708186 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.709470 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.709472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.709861 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.709862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.709970 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.710644 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:10:31.710645 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:31.710674 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:10:31.710696 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:31.710866 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:31.710869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:31.710871 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.710918 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.710995 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":102,"threads_started":1}
I20260923 15:10:31.711974 28406 heartbeater.cc:347] Connected to a master server at 127.7.250.126:37505
I20260923 15:10:31.712107 28406 heartbeater.cc:464] Registering TS with master...
I20260923 15:10:31.712308 28406 heartbeater.cc:511] Master 127.7.250.126:37505 requested a full tablet report, sending...
I20260923 15:10:31.712738 27952 ts_manager.cc:194] Registered new tserver with Master: 6d8ecd60897a49588d379b3e125dacc0 (127.7.250.67:35185)
I20260923 15:10:31.713141 27952 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.250.67:34429
I20260923 15:10:31.720359 8169 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:10:31.725026 28010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.721940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35266 (local address 127.7.250.126:37505)
0923 15:10:31.722018 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.722021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.722129 (+ 108us) server_negotiation.cc:407] Connection header received
0923 15:10:31.722244 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.722245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.722305 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.722369 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:31.722638 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.723168 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.723984 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.724181 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.724244 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.724823 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:31.724824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:31.724824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:31.724828 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:31.724841 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:31.724845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:31.724888 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.724946 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.724969 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:10:31.726979 27952 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:10:31.728513 27952 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:10:31.736794 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.731960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43631 (local address 127.0.0.1:58874)
0923 15:10:31.732554 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.732564 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.732684 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.732891 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.732892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.732894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.732953 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.732954 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.734171 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.734172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.735450 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.735452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.736445 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.736446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.736538 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.736553 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.736637 (+ 84us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.736695 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":53,"thread_start_us":39,"threads_started":1}
I20260923 15:10:31.736793 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.732372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58874 (local address 127.7.250.65:43631)
0923 15:10:31.732605 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.732610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.732625 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:10:31.732788 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.732792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.732807 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.732879 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.733022 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.734087 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.735548 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.736379 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.736445 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.736540 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.736662 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.736702 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260923 15:10:31.736922 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.731960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.66:43471 (local address 127.0.0.1:59240)
0923 15:10:31.732116 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.732162 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.732232 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.732474 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.732477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.732485 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.732627 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.732630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.733978 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.733979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.735449 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.735452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.736465 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.736466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.736544 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.736566 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.736599 (+ 33us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.736660 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":3,"thread_start_us":68,"threads_started":1}
I20260923 15:10:31.737313 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.732098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59240 (local address 127.7.250.66:43471)
0923 15:10:31.732290 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.732295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.732304 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:31.732343 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.732345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.732387 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.732471 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.732718 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.733888 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.735638 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.736339 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.736750 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.737157 (+ 407us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.737187 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.737232 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":87,"threads_started":1}
I20260923 15:10:31.737829 28075 tablet_service.cc:1467] Processing CreateTablet for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b (DEFAULT_TABLE table=test-table [id=5f1329f0b9694e659c032009b0d57f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:31.738230 28075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:31.739199 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.732288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58594 (local address 127.7.250.67:35185)
0923 15:10:31.734008 (+ 1720us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.734012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.734022 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:10:31.734058 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.734061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.734078 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.734167 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.735681 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.736632 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.738353 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.738847 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.738908 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.738963 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.739107 (+ 144us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.739139 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260923 15:10:31.739198 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.732197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:35185 (local address 127.0.0.1:58594)
0923 15:10:31.732670 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.732681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.732737 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.735564 (+ 2827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.735565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.735568 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.735613 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.735615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.736725 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.736726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.738231 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.738234 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.738943 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.738944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.739012 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.739017 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.739089 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.739129 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":95,"threads_started":1}
I20260923 15:10:31.739959 28340 tablet_service.cc:1467] Processing CreateTablet for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b (DEFAULT_TABLE table=test-table [id=5f1329f0b9694e659c032009b0d57f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:31.740316 28340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:31.741966 28420 tablet_bootstrap.cc:492] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: Bootstrap starting.
I20260923 15:10:31.742719 28420 tablet_bootstrap.cc:654] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:31.742702 28208 tablet_service.cc:1467] Processing CreateTablet for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b (DEFAULT_TABLE table=test-table [id=5f1329f0b9694e659c032009b0d57f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:10:31.743058 28208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:31.743088 28420 log.cc:824] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:31.743963 28420 tablet_bootstrap.cc:492] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: No bootstrap required, opened a new log
I20260923 15:10:31.744046 28420 ts_tablet_manager.cc:1397] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:10:31.744165 28422 tablet_bootstrap.cc:492] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Bootstrap starting.
I20260923 15:10:31.744751 28422 tablet_bootstrap.cc:654] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:31.745075 28422 log.cc:824] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:31.745821 28422 tablet_bootstrap.cc:492] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: No bootstrap required, opened a new log
I20260923 15:10:31.745908 28422 ts_tablet_manager.cc:1397] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:10:31.746572 28424 tablet_bootstrap.cc:492] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Bootstrap starting.
I20260923 15:10:31.746694 28420 raft_consensus.cc:348] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.746891 28420 raft_consensus.cc:374] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:31.746935 28420 raft_consensus.cc:729] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183ca4e897fd457d9fe4cb4a71e072e8, State: Initialized, Role: FOLLOWER
I20260923 15:10:31.747072 28420 consensus_queue.cc:260] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.747340 28420 ts_tablet_manager.cc:1428] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:10:31.747360 28142 heartbeater.cc:503] Master 127.7.250.126:37505 was elected leader, sending a full tablet report...
I20260923 15:10:31.747697 28424 tablet_bootstrap.cc:654] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:31.748093 28424 log.cc:824] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:31.748951 28424 tablet_bootstrap.cc:492] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: No bootstrap required, opened a new log
I20260923 15:10:31.748848 28422 raft_consensus.cc:348] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.749029 28424 ts_tablet_manager.cc:1397] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:10:31.749022 28422 raft_consensus.cc:374] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:31.749063 28422 raft_consensus.cc:729] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d8ecd60897a49588d379b3e125dacc0, State: Initialized, Role: FOLLOWER
I20260923 15:10:31.749200 28422 consensus_queue.cc:260] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.749468 28422 ts_tablet_manager.cc:1428] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260923 15:10:31.749605 28406 heartbeater.cc:503] Master 127.7.250.126:37505 was elected leader, sending a full tablet report...
I20260923 15:10:31.751794 28424 raft_consensus.cc:348] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.751924 28424 raft_consensus.cc:374] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:31.751947 28424 raft_consensus.cc:729] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d5f20ee780d4a4a91ab0f99f74ce9b4, State: Initialized, Role: FOLLOWER
I20260923 15:10:31.752084 28424 consensus_queue.cc:260] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.752373 28424 ts_tablet_manager.cc:1428] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:10:31.752419 28274 heartbeater.cc:503] Master 127.7.250.126:37505 was elected leader, sending a full tablet report...
W20260923 15:10:31.818677 28275 tablet.cc:2369] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:10:31.827493 28428 raft_consensus.cc:482] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:10:31.827616 28428 raft_consensus.cc:504] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.827941 28428 leader_election.cc:288] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183ca4e897fd457d9fe4cb4a71e072e8 (127.7.250.65:43631), 6d8ecd60897a49588d379b3e125dacc0 (127.7.250.67:35185)
I20260923 15:10:31.830922 28418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.828214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:40753 (local address 127.7.250.67:35185)
0923 15:10:31.828263 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.828265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.828273 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:10:31.828305 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.828306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.828317 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.828388 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.828577 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.829103 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.830148 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.830579 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.830665 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.830674 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.830824 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.830859 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 15:10:31.830933 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.828134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.67:35185 (local address 127.7.250.66:40753)
0923 15:10:31.828177 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.828188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.828230 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.828432 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.828433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.828435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.828511 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.828513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.829201 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.829202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.830054 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.830056 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.830664 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.830665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.830741 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.830744 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.830806 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.830847 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260923 15:10:31.831202 28419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.828203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.250.66:45189 (local address 127.7.250.65:43631)
0923 15:10:31.828251 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.828254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.828427 (+ 173us) server_negotiation.cc:407] Connection header received
0923 15:10:31.828496 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.828497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.828510 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.828633 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:10:31.828763 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.829341 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.830461 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.830817 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.830906 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.830914 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.831098 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.831138 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:10:31.831287 28360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8618eb7e2a0c41aaa7fc0616c8d23e3b" candidate_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8ecd60897a49588d379b3e125dacc0" is_pre_election: true
I20260923 15:10:31.831410 28429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.828134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43631 (local address 127.7.250.66:45189)
0923 15:10:31.828347 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.828357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:31.828425 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:31.828636 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:31.828637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:31.828638 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:10:31.828693 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.828694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.829418 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.829419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.830373 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:31.830376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.830922 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.830924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:31.830991 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.830994 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:10:31.831286 (+ 292us) client_negotiation.cc:240] Negotiation successful
0923 15:10:31.831326 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":53,"thread_start_us":97,"threads_started":1}
I20260923 15:10:31.831460 28360 raft_consensus.cc:2454] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d5f20ee780d4a4a91ab0f99f74ce9b4 in term 0.
I20260923 15:10:31.831758 28095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8618eb7e2a0c41aaa7fc0616c8d23e3b" candidate_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" is_pre_election: true
I20260923 15:10:31.831804 28160 leader_election.cc:302] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d8ecd60897a49588d379b3e125dacc0, 7d5f20ee780d4a4a91ab0f99f74ce9b4; no voters:
I20260923 15:10:31.831928 28095 raft_consensus.cc:2454] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d5f20ee780d4a4a91ab0f99f74ce9b4 in term 0.
I20260923 15:10:31.831919 28428 raft_consensus.cc:2781] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:10:31.831974 28428 raft_consensus.cc:482] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:10:31.832010 28428 raft_consensus.cc:3037] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.832778 28428 raft_consensus.cc:504] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.832933 28428 leader_election.cc:288] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [CANDIDATE]: Term 1 election: Requested vote from peers 183ca4e897fd457d9fe4cb4a71e072e8 (127.7.250.65:43631), 6d8ecd60897a49588d379b3e125dacc0 (127.7.250.67:35185)
I20260923 15:10:31.833055 28095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8618eb7e2a0c41aaa7fc0616c8d23e3b" candidate_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183ca4e897fd457d9fe4cb4a71e072e8"
I20260923 15:10:31.833127 28095 raft_consensus.cc:3037] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.833163 28360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8618eb7e2a0c41aaa7fc0616c8d23e3b" candidate_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d8ecd60897a49588d379b3e125dacc0"
I20260923 15:10:31.833240 28360 raft_consensus.cc:3037] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:31.833878 28095 raft_consensus.cc:2454] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d5f20ee780d4a4a91ab0f99f74ce9b4 in term 1.
I20260923 15:10:31.834059 28360 raft_consensus.cc:2454] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d5f20ee780d4a4a91ab0f99f74ce9b4 in term 1.
I20260923 15:10:31.834127 28162 leader_election.cc:302] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183ca4e897fd457d9fe4cb4a71e072e8, 7d5f20ee780d4a4a91ab0f99f74ce9b4; no voters:
I20260923 15:10:31.834227 28428 raft_consensus.cc:2781] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:31.834412 28428 raft_consensus.cc:686] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 1 LEADER]: Becoming Leader. State: Replica: 7d5f20ee780d4a4a91ab0f99f74ce9b4, State: Running, Role: LEADER
I20260923 15:10:31.834517 28428 consensus_queue.cc:237] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } }
I20260923 15:10:31.835389 27952 catalog_manager.cc:5495] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d5f20ee780d4a4a91ab0f99f74ce9b4 (127.7.250.66). New cstate: current_term: 1 leader_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d5f20ee780d4a4a91ab0f99f74ce9b4" member_type: VOTER last_known_addr { host: "127.7.250.66" port: 43471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 } health_report { overall_health: UNKNOWN } } }
I20260923 15:10:31.852033 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.849435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.7.250.66:43471)
0923 15:10:31.849505 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:31.849507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:31.849519 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:10:31.849612 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:31.849613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:31.849633 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:31.849713 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:31.849847 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.850379 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.851196 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:31.851426 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:31.851499 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:31.851662 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:31.851769 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:31.851854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:31.851892 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 15:10:31.851930 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:10:31.853885 28095 raft_consensus.cc:1261] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8 [term 1 FOLLOWER]: Refusing update from remote peer 7d5f20ee780d4a4a91ab0f99f74ce9b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:31.853885 28360 raft_consensus.cc:1261] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 1 FOLLOWER]: Refusing update from remote peer 7d5f20ee780d4a4a91ab0f99f74ce9b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:31.854279 28430 consensus_queue.cc:1035] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "183ca4e897fd457d9fe4cb4a71e072e8" member_type: VOTER last_known_addr { host: "127.7.250.65" port: 43631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:31.854422 28428 consensus_queue.cc:1035] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d8ecd60897a49588d379b3e125dacc0" member_type: VOTER last_known_addr { host: "127.7.250.67" port: 35185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:31.856698 28433 mvcc.cc:205] Tried to move back new op lower bound from 7332561845670465536 to 7332561845594951680. Current Snapshot: MvccSnapshot[applied={T|T < 7332561845670465536}]
I20260923 15:10:31.857167 28434 mvcc.cc:205] Tried to move back new op lower bound from 7332561845670465536 to 7332561845594951680. Current Snapshot: MvccSnapshot[applied={T|T < 7332561845670465536}]
I20260923 15:10:31.857662 28435 mvcc.cc:205] Tried to move back new op lower bound from 7332561845670465536 to 7332561845594951680. Current Snapshot: MvccSnapshot[applied={T|T < 7332561845670465536 or (T in {7332561845670465536})}]
I20260923 15:10:31.859877 27951 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:31.859979 27951 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:10:31.860848 27951 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eceb7004a8c54710bbb69de767f17bc7: Sending DeleteTablet for 3 replicas of tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b
I20260923 15:10:31.861217 28075 tablet_service.cc:1514] Processing DeleteTablet for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:31 UTC) from {username='slave'} at 127.0.0.1:58874
I20260923 15:10:31.861207 28208 tablet_service.cc:1514] Processing DeleteTablet for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:31 UTC) from {username='slave'} at 127.0.0.1:59240
I20260923 15:10:31.861354 28340 tablet_service.cc:1514] Processing DeleteTablet for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:10:31 UTC) from {username='slave'} at 127.0.0.1:58594
I20260923 15:10:31.861661 28443 tablet_replica.cc:330] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 183ca4e897fd457d9fe4cb4a71e072e8: stopping tablet replica
I20260923 15:10:31.861686 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28012
I20260923 15:10:31.863193 28444 tablet_replica.cc:330] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: stopping tablet replica
I20260923 15:10:31.863296 28444 raft_consensus.cc:2229] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:10:31.863440 28444 raft_consensus.cc:2258] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:31.863634 28445 tablet_replica.cc:330] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: stopping tablet replica
I20260923 15:10:31.863739 28445 raft_consensus.cc:2229] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:10:31.863838 28445 raft_consensus.cc:2258] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:10:31.864446 28445 ts_tablet_manager.cc:1905] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:31.864445 28444 ts_tablet_manager.cc:1905] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:10:31.866161 28445 ts_tablet_manager.cc:1918] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:31.866230 28445 log.cc:1196] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-2/wal/wals/8618eb7e2a0c41aaa7fc0616c8d23e3b
I20260923 15:10:31.866523 28445 ts_tablet_manager.cc:1939] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 6d8ecd60897a49588d379b3e125dacc0: Deleting consensus metadata
I20260923 15:10:31.866529 28444 ts_tablet_manager.cc:1918] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:10:31.866608 28444 log.cc:1196] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Deleting WAL directory at /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175983307587-8169-0/minicluster-data/ts-1/wal/wals/8618eb7e2a0c41aaa7fc0616c8d23e3b
I20260923 15:10:31.866848 28444 ts_tablet_manager.cc:1939] T 8618eb7e2a0c41aaa7fc0616c8d23e3b P 7d5f20ee780d4a4a91ab0f99f74ce9b4: Deleting consensus metadata
I20260923 15:10:31.866976 27936 catalog_manager.cc:4841] TS 6d8ecd60897a49588d379b3e125dacc0 (127.7.250.67:35185): tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b (table test-table [id=5f1329f0b9694e659c032009b0d57f09]) successfully deleted
I20260923 15:10:31.867329 27936 catalog_manager.cc:4841] TS 7d5f20ee780d4a4a91ab0f99f74ce9b4 (127.7.250.66:43471): tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b (table test-table [id=5f1329f0b9694e659c032009b0d57f09]) successfully deleted
W20260923 15:10:31.868872 27938 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.250.65:43631 (error 108)
I20260923 15:10:31.869410 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:31.869261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.65:43631 (local address 127.0.0.1:58876)
0923 15:10:31.869325 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:31.869370 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:10:31.869431 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28145
W20260923 15:10:31.869534 27938 catalog_manager.cc:4584] TS 183ca4e897fd457d9fe4cb4a71e072e8 (127.7.250.65:43631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8618eb7e2a0c41aaa7fc0616c8d23e3b: Network error: Client connection negotiation failed: client connection to 127.7.250.65:43631: connect: Connection refused (error 111)
I20260923 15:10:31.874162 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28277
I20260923 15:10:31.878659 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 27921
2026-09-23T15:10:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (778 ms)
[----------] 6 tests from EncryptionPolicyTest (8398 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:31 dist-test-slave-bb3x krb5kdc[28449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:31 dist-test-slave-bb3x krb5kdc[28449](info): set up 2 sockets
Sep 23 15:10:31 dist-test-slave-bb3x krb5kdc[28449](info): commencing operation
krb5kdc: starting...
W20260923 15:10:32.951918 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:33 dist-test-slave-bb3x krb5kdc[28449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:33.077155 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34749
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:33.161576 28462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:33.161803 28462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:33.161828 28462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:33.162992 28462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:33.163033 28462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:33.163048 28462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:33.163060 28462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:33.163072 28462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:33.164517 28462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:33.164726 28462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:33.164899 28462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:33.170303 28467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:33.170257 28470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:33.170300 28468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:33.170678 28462 server_base.cc:1034] running on GCE node
I20260923 15:10:33.170895 28462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:33.171147 28462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:33.172293 28462 hybrid_clock.cc:648] HybridClock initialized: now 1790176233172279 us; error 28 us; skew 500 ppm
Sep 23 15:10:33 dist-test-slave-bb3x krb5kdc[28449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176233, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:33.175076 28462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:33.175570 28462 webserver.cc:460] Webserver started at http://127.0.0.1:35791/ using document root <none> and password file <none>
I20260923 15:10:33.175818 28462 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:33.175865 28462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:33.175987 28462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:33.177053 28462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7"
format_stamp: "Formatted at 2026-09-23 15:10:33 on dist-test-slave-bb3x"
server_key: "fccb4e889210811c7ec18203fd7151f6"
server_key_iv: "540e8c870cfb04ba7332852a4cbf4b25"
server_key_version: "encryptionkey@0"
I20260923 15:10:33.177412 28462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7"
format_stamp: "Formatted at 2026-09-23 15:10:33 on dist-test-slave-bb3x"
server_key: "fccb4e889210811c7ec18203fd7151f6"
server_key_iv: "540e8c870cfb04ba7332852a4cbf4b25"
server_key_version: "encryptionkey@0"
I20260923 15:10:33.178854 28462 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:33.179636 28478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:33.179860 28462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:33.179935 28462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7"
format_stamp: "Formatted at 2026-09-23 15:10:33 on dist-test-slave-bb3x"
server_key: "fccb4e889210811c7ec18203fd7151f6"
server_key_iv: "540e8c870cfb04ba7332852a4cbf4b25"
server_key_version: "encryptionkey@0"
I20260923 15:10:33.180027 28462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:33.188088 28462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:33.190645 28462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:33.190811 28462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:33.195115 28462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34749
I20260923 15:10:33.195156 28530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34749 every 8 connection(s)
I20260923 15:10:33.195524 28462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:33.196655 28531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:33.199487 28531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7: Bootstrap starting.
I20260923 15:10:33.200181 28531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:33.200531 28531 log.cc:824] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:33.201350 28531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7: No bootstrap required, opened a new log
I20260923 15:10:33.202369 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28462
I20260923 15:10:33.202494 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:33.202692 8169 external_mini_cluster.cc:1428] Setting key d6e164a2b83aab3654eba829d75b7bdc
I20260923 15:10:33.204164 28531 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260923 15:10:33.204371 28531 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:33.204429 28531 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c06dd99baa4ae9b4ad3d949b04a4a7, State: Initialized, Role: FOLLOWER
I20260923 15:10:33.204582 28531 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260923 15:10:33.204681 28531 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:33.204736 28531 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:33.204795 28531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:33.206110 28531 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260923 15:10:33.206244 28531 leader_election.cc:302] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0c06dd99baa4ae9b4ad3d949b04a4a7; no voters:
I20260923 15:10:33.206455 28531 leader_election.cc:288] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:33.206512 28536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:33.206673 28536 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [term 1 LEADER]: Becoming Leader. State: Replica: a0c06dd99baa4ae9b4ad3d949b04a4a7, State: Running, Role: LEADER
I20260923 15:10:33.206728 28531 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:33.206774 28536 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } }
I20260923 15:10:33.207116 28536 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0c06dd99baa4ae9b4ad3d949b04a4a7. Latest consensus state: current_term: 1 leader_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } } }
I20260923 15:10:33.207175 28536 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:33.207286 28538 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0c06dd99baa4ae9b4ad3d949b04a4a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34749 } } }
I20260923 15:10:33.207352 28538 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:33.208160 28542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:33.208698 28542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:33.210322 28542 catalog_manager.cc:1347] Generated new cluster ID: 3c31b66a1a7f40f584fe7463bca0f31b
I20260923 15:10:33.210379 28542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:10:33 dist-test-slave-bb3x krb5kdc[28449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:10:33.216259 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:33.203325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47533 (local address 127.0.0.1:34749)
0923 15:10:33.203553 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:33.203566 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:33.203609 (+ 43us) server_negotiation.cc:407] Connection header received
0923 15:10:33.204110 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:33.204119 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:33.204195 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:33.204386 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:33.205190 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:33.206873 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:33.208284 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:33.208944 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:33.209100 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:33.212157 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:33.212162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:33.212162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:33.212176 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:33.214221 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:33.214773 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:33.214776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:33.214777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:33.214921 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:33.215211 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:33.215212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:33.215212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:33.215529 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:33.215743 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:33.215889 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 15:10:33.216017 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
I20260923 15:10:33.217439 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:33.226818 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:33.219028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41101 (local address 127.0.0.1:34749)
0923 15:10:33.219075 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:33.219078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:33.219227 (+ 149us) server_negotiation.cc:407] Connection header received
0923 15:10:33.219319 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:33.219320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:33.219332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:33.219388 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:33.220000 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:33.220577 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:33.221419 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:33.221697 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:33.221800 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:33.223308 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:33.223310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:33.223310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:33.223318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:33.225080 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:33.225819 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:33.225821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:33.225821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:33.225893 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:33.226159 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:33.226160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:33.226160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:33.226345 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:33.226575 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:33.226629 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 15:10:33.226694 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:10:33.231134 28542 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:33.231842 28542 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:33.240639 28542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0c06dd99baa4ae9b4ad3d949b04a4a7: Generated new TSK 0
I20260923 15:10:33.240875 28542 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:33.243171 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28462
W20260923 15:10:33.248732 28447 connection.cc:535] client connection to 127.0.0.1:34749 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34749): Connection reset by peer (error 104)
2026-09-23T15:10:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1364 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:33 dist-test-slave-bb3x krb5kdc[28562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:33 dist-test-slave-bb3x krb5kdc[28562](info): set up 2 sockets
Sep 23 15:10:33 dist-test-slave-bb3x krb5kdc[28562](info): commencing operation
krb5kdc: starting...
W20260923 15:10:34.283908 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:34 dist-test-slave-bb3x krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176234, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:34.411489 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:34.493453 28575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:34.493688 28575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:34.493726 28575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:34.494933 28575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:34.494997 28575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:34.495020 28575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:34.495046 28575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:34.495077 28575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:34.496541 28575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:34.496810 28575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:34.497041 28575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:34.502156 28581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:34.502162 28580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:34.502177 28583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:34.502426 28575 server_base.cc:1034] running on GCE node
I20260923 15:10:34.502676 28575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:34.502985 28575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:34.504163 28575 hybrid_clock.cc:648] HybridClock initialized: now 1790176234504141 us; error 36 us; skew 500 ppm
Sep 23 15:10:34 dist-test-slave-bb3x krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176234, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:34.506896 28575 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:34.507475 28575 webserver.cc:460] Webserver started at http://127.0.0.1:34841/ using document root <none> and password file <none>
I20260923 15:10:34.507730 28575 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:34.507808 28575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:34.507966 28575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:34.509016 28575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "b57b98522edb4fb3805bbce956253c95"
format_stamp: "Formatted at 2026-09-23 15:10:34 on dist-test-slave-bb3x"
server_key: "5960f65ce011f7bc3d40bab9bcf2d761"
server_key_iv: "a8f413982016d21be0a2aa8728ed9532"
server_key_version: "encryptionkey@0"
I20260923 15:10:34.509379 28575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "b57b98522edb4fb3805bbce956253c95"
format_stamp: "Formatted at 2026-09-23 15:10:34 on dist-test-slave-bb3x"
server_key: "5960f65ce011f7bc3d40bab9bcf2d761"
server_key_iv: "a8f413982016d21be0a2aa8728ed9532"
server_key_version: "encryptionkey@0"
I20260923 15:10:34.510841 28575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:34.511731 28590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:34.511929 28575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:34.512071 28575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "b57b98522edb4fb3805bbce956253c95"
format_stamp: "Formatted at 2026-09-23 15:10:34 on dist-test-slave-bb3x"
server_key: "5960f65ce011f7bc3d40bab9bcf2d761"
server_key_iv: "a8f413982016d21be0a2aa8728ed9532"
server_key_version: "encryptionkey@0"
I20260923 15:10:34.512171 28575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:34.536912 28575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:34.539239 28575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:34.539429 28575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:34.544147 28575 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43799
I20260923 15:10:34.544184 28643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43799 every 8 connection(s)
I20260923 15:10:34.544595 28575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:34.545750 28644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:34.546617 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28575
I20260923 15:10:34.546758 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:34.546989 8169 external_mini_cluster.cc:1428] Setting key 734adc76ca3bdd96176a909396d8fd4b
I20260923 15:10:34.549413 28644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95: Bootstrap starting.
I20260923 15:10:34.550192 28644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:34.550592 28644 log.cc:824] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:34.551412 28644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95: No bootstrap required, opened a new log
I20260923 15:10:34.553957 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:34.547627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39429 (local address 127.0.0.1:43799)
0923 15:10:34.547845 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:34.547856 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:34.547891 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:10:34.548471 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:34.548482 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:34.548555 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:34.548719 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:34.548911 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:34.550437 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:34.551557 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:34.552227 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:34.552357 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:34.553132 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:34.553136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:34.553137 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:34.553388 (+ 251us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:34.553438 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:34.553458 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:34.553566 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:34.553686 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 15:10:34.553724 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260923 15:10:34.554076 28644 raft_consensus.cc:348] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b98522edb4fb3805bbce956253c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43799 } }
I20260923 15:10:34.554255 28644 raft_consensus.cc:374] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:34.554319 28644 raft_consensus.cc:729] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b57b98522edb4fb3805bbce956253c95, State: Initialized, Role: FOLLOWER
I20260923 15:10:34.554512 28644 consensus_queue.cc:260] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b98522edb4fb3805bbce956253c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43799 } }
I20260923 15:10:34.554610 28644 raft_consensus.cc:388] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:34.554666 28644 raft_consensus.cc:482] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:34.554720 28644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:34.555198 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:34.555490 28644 raft_consensus.cc:504] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b98522edb4fb3805bbce956253c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43799 } }
I20260923 15:10:34.555613 28644 leader_election.cc:302] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b57b98522edb4fb3805bbce956253c95; no voters:
I20260923 15:10:34.555838 28644 leader_election.cc:288] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:34.555871 28649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:34.556010 28649 raft_consensus.cc:686] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [term 1 LEADER]: Becoming Leader. State: Replica: b57b98522edb4fb3805bbce956253c95, State: Running, Role: LEADER
I20260923 15:10:34.556124 28644 sys_catalog.cc:564] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:34.556156 28649 consensus_queue.cc:237] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b98522edb4fb3805bbce956253c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43799 } }
I20260923 15:10:34.556531 28650 sys_catalog.cc:455] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b57b98522edb4fb3805bbce956253c95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b98522edb4fb3805bbce956253c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43799 } } }
I20260923 15:10:34.556639 28650 sys_catalog.cc:458] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:34.556514 28649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b57b98522edb4fb3805bbce956253c95. Latest consensus state: current_term: 1 leader_uuid: "b57b98522edb4fb3805bbce956253c95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b57b98522edb4fb3805bbce956253c95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43799 } } }
I20260923 15:10:34.556910 28649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:34.556890 28660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:34.557524 28660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:34.559291 28660 catalog_manager.cc:1347] Generated new cluster ID: eafa2276fd6c474786f43ee8279483e3
I20260923 15:10:34.559350 28660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:34.561324 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:34.556889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50145 (local address 127.0.0.1:43799)
0923 15:10:34.556931 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:34.556933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:34.557879 (+ 946us) server_negotiation.cc:407] Connection header received
0923 15:10:34.558002 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:34.558003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:34.558016 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:34.558074 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:34.558321 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:34.559022 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:34.559953 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:34.560198 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:34.560291 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:34.561048 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:34.561050 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:34.561050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:34.561056 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:34.561072 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:34.561077 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:34.561137 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:34.561231 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 15:10:34.561242 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:10:34.584944 28660 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:34.585518 28660 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:34.591296 28660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b57b98522edb4fb3805bbce956253c95: Generated new TSK 0
I20260923 15:10:34.591509 28660 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:34.594981 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28575
W20260923 15:10:34.599692 28560 connection.cc:535] client connection to 127.0.0.1:43799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43799): Connection reset by peer (error 104)
2026-09-23T15:10:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:34 dist-test-slave-bb3x krb5kdc[28675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:34 dist-test-slave-bb3x krb5kdc[28675](info): set up 2 sockets
Sep 23 15:10:34 dist-test-slave-bb3x krb5kdc[28675](info): commencing operation
krb5kdc: starting...
W20260923 15:10:35.660046 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:35 dist-test-slave-bb3x krb5kdc[28675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:35.786232 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37405
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:35.868340 28688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:35.868578 28688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:35.868604 28688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:35.869760 28688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:35.869797 28688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:35.869812 28688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:35.869825 28688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:35.869848 28688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:35.871332 28688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37405
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:35.871537 28688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:35.871708 28688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:35.876761 28693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:35.876780 28696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:35.876823 28694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:35.876863 28688 server_base.cc:1034] running on GCE node
I20260923 15:10:35.877254 28688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:35.877558 28688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:35.878721 28688 hybrid_clock.cc:648] HybridClock initialized: now 1790176235878720 us; error 42 us; skew 500 ppm
Sep 23 15:10:35 dist-test-slave-bb3x krb5kdc[28675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176235, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:35.891131 28688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:35.891680 28688 webserver.cc:460] Webserver started at http://127.0.0.1:37757/ using document root <none> and password file <none>
I20260923 15:10:35.891939 28688 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:35.891992 28688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:35.892138 28688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:35.893347 28688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "c050671c28594334b9fa65709688122f"
format_stamp: "Formatted at 2026-09-23 15:10:35 on dist-test-slave-bb3x"
server_key: "56d24a31e1c10fea5b2158689822f0ec"
server_key_iv: "f4835495a3c7b26f303322dff5f42f14"
server_key_version: "encryptionkey@0"
I20260923 15:10:35.893739 28688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "c050671c28594334b9fa65709688122f"
format_stamp: "Formatted at 2026-09-23 15:10:35 on dist-test-slave-bb3x"
server_key: "56d24a31e1c10fea5b2158689822f0ec"
server_key_iv: "f4835495a3c7b26f303322dff5f42f14"
server_key_version: "encryptionkey@0"
I20260923 15:10:35.895414 28688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:10:35.896454 28703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:35.896737 28688 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:35.896842 28688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "c050671c28594334b9fa65709688122f"
format_stamp: "Formatted at 2026-09-23 15:10:35 on dist-test-slave-bb3x"
server_key: "56d24a31e1c10fea5b2158689822f0ec"
server_key_iv: "f4835495a3c7b26f303322dff5f42f14"
server_key_version: "encryptionkey@0"
I20260923 15:10:35.896917 28688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:35.901065 28688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:35.903532 28688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:35.903702 28688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:35.908447 28688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37405
I20260923 15:10:35.908475 28755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37405 every 8 connection(s)
I20260923 15:10:35.908842 28688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:35.909926 28756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:35.911419 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28688
I20260923 15:10:35.911552 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:35.911775 8169 external_mini_cluster.cc:1428] Setting key 7cf8601bcbeb25c0710b7242b208dac6
I20260923 15:10:35.913054 28756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f: Bootstrap starting.
I20260923 15:10:35.913863 28756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:35.914196 28756 log.cc:824] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:35.914865 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:35.912353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59983 (local address 127.0.0.1:37405)
0923 15:10:35.912652 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:35.912663 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:35.912697 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:10:35.913303 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:35.913313 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:35.913349 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:35.913560 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:35.914416 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:35.914423 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:35.914423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:35.914433 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:35.914471 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:35.914482 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:35.914592 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:35.914634 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 15:10:35.914678 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
I20260923 15:10:35.915184 28756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f: No bootstrap required, opened a new log
I20260923 15:10:35.915879 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:35.917716 28756 raft_consensus.cc:348] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c050671c28594334b9fa65709688122f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37405 } }
I20260923 15:10:35.917896 28756 raft_consensus.cc:374] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:35.917948 28756 raft_consensus.cc:729] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c050671c28594334b9fa65709688122f, State: Initialized, Role: FOLLOWER
I20260923 15:10:35.918080 28756 consensus_queue.cc:260] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c050671c28594334b9fa65709688122f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37405 } }
I20260923 15:10:35.918180 28756 raft_consensus.cc:388] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:35.918229 28756 raft_consensus.cc:482] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:35.918306 28756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:35.919194 28756 raft_consensus.cc:504] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c050671c28594334b9fa65709688122f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37405 } }
I20260923 15:10:35.919322 28756 leader_election.cc:302] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c050671c28594334b9fa65709688122f; no voters:
I20260923 15:10:35.919483 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:35.917409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54069 (local address 127.0.0.1:37405)
0923 15:10:35.917463 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:35.917465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:35.918438 (+ 973us) server_negotiation.cc:407] Connection header received
0923 15:10:35.918539 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:35.918540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:35.918546 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:35.918603 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:35.919212 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:35.919213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:35.919214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:35.919217 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:35.919232 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:35.919235 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:35.919331 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:35.919403 (+ 72us) server_negotiation.cc:299] Negotiation successful
0923 15:10:35.919418 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:10:35.919535 28756 leader_election.cc:288] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:35.919636 28766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:35.919785 28766 raft_consensus.cc:686] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [term 1 LEADER]: Becoming Leader. State: Replica: c050671c28594334b9fa65709688122f, State: Running, Role: LEADER
I20260923 15:10:35.919824 28756 sys_catalog.cc:564] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:35.919886 28766 consensus_queue.cc:237] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c050671c28594334b9fa65709688122f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37405 } }
I20260923 15:10:35.920235 28766 sys_catalog.cc:455] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c050671c28594334b9fa65709688122f. Latest consensus state: current_term: 1 leader_uuid: "c050671c28594334b9fa65709688122f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c050671c28594334b9fa65709688122f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37405 } } }
I20260923 15:10:35.920248 28767 sys_catalog.cc:455] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c050671c28594334b9fa65709688122f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c050671c28594334b9fa65709688122f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37405 } } }
I20260923 15:10:35.920387 28766 sys_catalog.cc:458] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:35.920456 28767 sys_catalog.cc:458] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:35.920714 28774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:35.921231 28774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:35.922998 28774 catalog_manager.cc:1347] Generated new cluster ID: 97f94626b2854b93aec18fbcffac9831
I20260923 15:10:35.923056 28774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:35.934787 28774 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:35.934991 28774 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:35.946602 28774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c050671c28594334b9fa65709688122f: Generated new TSK 0
I20260923 15:10:35.946827 28774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:35.949365 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28688
2026-09-23T15:10:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1354 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:36 dist-test-slave-bb3x krb5kdc[28787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:36 dist-test-slave-bb3x krb5kdc[28787](info): set up 2 sockets
Sep 23 15:10:36 dist-test-slave-bb3x krb5kdc[28787](info): commencing operation
krb5kdc: starting...
W20260923 15:10:36.991870 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:37 dist-test-slave-bb3x krb5kdc[28787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:37.121829 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34817
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:37.205797 28800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:37.206076 28800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:37.206121 28800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:37.207465 28800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:37.207542 28800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:37.207569 28800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:37.207602 28800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:37.207636 28800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:37.209282 28800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:37.209594 28800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:37.209839 28800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:10:37.215296 28800 server_base.cc:1034] running on GCE node
W20260923 15:10:37.215298 28808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:37.215267 28806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:37.215267 28805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:37.215812 28800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:37.216063 28800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:37.217231 28800 hybrid_clock.cc:648] HybridClock initialized: now 1790176237217236 us; error 47 us; skew 500 ppm
Sep 23 15:10:37 dist-test-slave-bb3x krb5kdc[28787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176237, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:37.219920 28800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:37.220371 28800 webserver.cc:460] Webserver started at http://127.0.0.1:44395/ using document root <none> and password file <none>
I20260923 15:10:37.220631 28800 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:37.220703 28800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:37.220850 28800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:37.221969 28800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "2cdba1f855b94e1bb46ae90eaf481420"
format_stamp: "Formatted at 2026-09-23 15:10:37 on dist-test-slave-bb3x"
server_key: "4fe555563b8a3259de8e339ff7d1c077"
server_key_iv: "cc6d5aa199adfb47181ade17a5e1b7a5"
server_key_version: "encryptionkey@0"
I20260923 15:10:37.222422 28800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "2cdba1f855b94e1bb46ae90eaf481420"
format_stamp: "Formatted at 2026-09-23 15:10:37 on dist-test-slave-bb3x"
server_key: "4fe555563b8a3259de8e339ff7d1c077"
server_key_iv: "cc6d5aa199adfb47181ade17a5e1b7a5"
server_key_version: "encryptionkey@0"
I20260923 15:10:37.223810 28800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:37.224622 28815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:37.224828 28800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:10:37.224928 28800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "2cdba1f855b94e1bb46ae90eaf481420"
format_stamp: "Formatted at 2026-09-23 15:10:37 on dist-test-slave-bb3x"
server_key: "4fe555563b8a3259de8e339ff7d1c077"
server_key_iv: "cc6d5aa199adfb47181ade17a5e1b7a5"
server_key_version: "encryptionkey@0"
I20260923 15:10:37.225014 28800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:37.239068 28800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:37.241578 28800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:37.241775 28800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:37.246080 28800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34817
I20260923 15:10:37.246116 28867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34817 every 8 connection(s)
I20260923 15:10:37.246500 28800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:37.246991 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28800
I20260923 15:10:37.247141 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:37.247372 8169 external_mini_cluster.cc:1428] Setting key 65cf7f7c11a01873f4a419b5ddfbea5d
I20260923 15:10:37.247771 28868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:37.251313 28868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420: Bootstrap starting.
I20260923 15:10:37.251986 28868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:37.252354 28868 log.cc:824] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:37.253151 28868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420: No bootstrap required, opened a new log
Sep 23 15:10:37 dist-test-slave-bb3x krb5kdc[28787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:10:37.255312 28868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdba1f855b94e1bb46ae90eaf481420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34817 } }
I20260923 15:10:37.255539 28868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:37.255582 28868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cdba1f855b94e1bb46ae90eaf481420, State: Initialized, Role: FOLLOWER
I20260923 15:10:37.255728 28868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdba1f855b94e1bb46ae90eaf481420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34817 } }
I20260923 15:10:37.255795 28868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:37.255816 28868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:37.255882 28868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:37.256645 28868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdba1f855b94e1bb46ae90eaf481420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34817 } }
I20260923 15:10:37.256775 28868 leader_election.cc:302] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cdba1f855b94e1bb46ae90eaf481420; no voters:
I20260923 15:10:37.256971 28868 leader_election.cc:288] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:37.257079 28873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:37.257247 28868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:37.257287 28873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [term 1 LEADER]: Becoming Leader. State: Replica: 2cdba1f855b94e1bb46ae90eaf481420, State: Running, Role: LEADER
I20260923 15:10:37.257445 28873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdba1f855b94e1bb46ae90eaf481420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34817 } }
I20260923 15:10:37.257871 28874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cdba1f855b94e1bb46ae90eaf481420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdba1f855b94e1bb46ae90eaf481420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34817 } } }
I20260923 15:10:37.257953 28874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:37.258175 28875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cdba1f855b94e1bb46ae90eaf481420. Latest consensus state: current_term: 1 leader_uuid: "2cdba1f855b94e1bb46ae90eaf481420" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cdba1f855b94e1bb46ae90eaf481420" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34817 } } }
I20260923 15:10:37.258280 28875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:37.258654 28882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:37.259660 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:37.247916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56877 (local address 127.0.0.1:34817)
0923 15:10:37.248103 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:37.248111 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:37.248136 (+ 25us) server_negotiation.cc:407] Connection header received
0923 15:10:37.248600 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:37.248607 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:37.248677 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:37.248800 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:37.249658 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:37.250652 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:37.251683 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:37.252105 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:37.252206 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:37.254867 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:37.254872 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:37.254873 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:37.254886 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:37.257584 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:37.258098 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:37.258102 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:37.258102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:37.258221 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:37.258483 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:37.258484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:37.258485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:37.258854 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:37.259037 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:37.259402 (+ 365us) server_negotiation.cc:299] Negotiation successful
0923 15:10:37.259513 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260923 15:10:37.260027 28882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:37.260723 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:37.261638 28882 catalog_manager.cc:1347] Generated new cluster ID: 73dd236a32e4467298d903458f2992f8
I20260923 15:10:37.261679 28882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:37.269001 28871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:37.262235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47411 (local address 127.0.0.1:34817)
0923 15:10:37.262289 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:37.262291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:37.262398 (+ 107us) server_negotiation.cc:407] Connection header received
0923 15:10:37.262447 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:37.262448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:37.262456 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:37.262544 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:37.263164 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:37.263659 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:37.264624 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:37.264812 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:37.264866 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:37.266128 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:37.266129 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:37.266130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:37.266137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:37.267754 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:37.268159 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:37.268160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:37.268160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:37.268224 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:37.268428 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:37.268429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:37.268430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:37.268580 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:37.268679 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:37.268819 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 15:10:37.268904 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:10:37.274123 28882 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:37.274893 28882 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:37.283041 28882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cdba1f855b94e1bb46ae90eaf481420: Generated new TSK 0
I20260923 15:10:37.283245 28882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:37.291940 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28800
2026-09-23T15:10:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1342 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:37 dist-test-slave-bb3x krb5kdc[28899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:37 dist-test-slave-bb3x krb5kdc[28899](info): set up 2 sockets
Sep 23 15:10:37 dist-test-slave-bb3x krb5kdc[28899](info): commencing operation
krb5kdc: starting...
W20260923 15:10:38.327950 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:38 dist-test-slave-bb3x krb5kdc[28899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176238, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:38.457562 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32843
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:38.539503 28912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:38.539763 28912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:38.539806 28912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:38.541010 28912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:38.541082 28912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:38.541107 28912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:38.541137 28912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:38.541167 28912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:38.542786 28912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:38.543092 28912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:38.543349 28912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:38.548725 28918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:38.548753 28917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:38.548767 28920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:38.548945 28912 server_base.cc:1034] running on GCE node
I20260923 15:10:38.549268 28912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:38.549527 28912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:38.550709 28912 hybrid_clock.cc:648] HybridClock initialized: now 1790176238550705 us; error 36 us; skew 500 ppm
Sep 23 15:10:38 dist-test-slave-bb3x krb5kdc[28899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176238, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:38.553294 28912 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:38.553784 28912 webserver.cc:460] Webserver started at http://127.0.0.1:40745/ using document root <none> and password file <none>
I20260923 15:10:38.554044 28912 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:38.554113 28912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:38.554306 28912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:38.555377 28912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "16ce46b1d08047c28878bebcb0d58f52"
format_stamp: "Formatted at 2026-09-23 15:10:38 on dist-test-slave-bb3x"
server_key: "c8ab28e34d90d29205d19aa6a59f52bb"
server_key_iv: "c12b400478262e1997e19dfdae979e92"
server_key_version: "encryptionkey@0"
I20260923 15:10:38.555751 28912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "16ce46b1d08047c28878bebcb0d58f52"
format_stamp: "Formatted at 2026-09-23 15:10:38 on dist-test-slave-bb3x"
server_key: "c8ab28e34d90d29205d19aa6a59f52bb"
server_key_iv: "c12b400478262e1997e19dfdae979e92"
server_key_version: "encryptionkey@0"
I20260923 15:10:38.557133 28912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:10:38.558017 28927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:38.558245 28912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:38.558377 28912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "16ce46b1d08047c28878bebcb0d58f52"
format_stamp: "Formatted at 2026-09-23 15:10:38 on dist-test-slave-bb3x"
server_key: "c8ab28e34d90d29205d19aa6a59f52bb"
server_key_iv: "c12b400478262e1997e19dfdae979e92"
server_key_version: "encryptionkey@0"
I20260923 15:10:38.558509 28912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:38.572450 28912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:38.575129 28912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:38.575335 28912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:38.579983 28912 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32843
I20260923 15:10:38.580058 28980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32843 every 8 connection(s)
I20260923 15:10:38.580457 28912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:38.581636 28981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:38.583025 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 28912
I20260923 15:10:38.583168 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:38.583367 8169 external_mini_cluster.cc:1428] Setting key e28102c967baf8b82ffbb08c8fb57891
I20260923 15:10:38.584828 28981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52: Bootstrap starting.
I20260923 15:10:38.585619 28981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:38.585952 28981 log.cc:824] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:38.586773 28981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52: No bootstrap required, opened a new log
I20260923 15:10:38.588969 28981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16ce46b1d08047c28878bebcb0d58f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32843 } }
I20260923 15:10:38.589154 28981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:38.589195 28981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16ce46b1d08047c28878bebcb0d58f52, State: Initialized, Role: FOLLOWER
I20260923 15:10:38.589324 28981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16ce46b1d08047c28878bebcb0d58f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32843 } }
I20260923 15:10:38.589444 28981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:38.589498 28981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:38.589542 28981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:38.589850 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:38.583932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57839 (local address 127.0.0.1:32843)
0923 15:10:38.584141 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:38.584151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:38.584185 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:10:38.584728 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:38.584736 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:38.584804 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:38.584964 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:38.585229 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:38.586690 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:38.587517 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:38.588148 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:38.588288 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:38.589006 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:38.589010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:38.589010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:38.589219 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:38.589257 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:38.589271 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:38.589362 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:38.589586 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 15:10:38.589666 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260923 15:10:38.590221 28981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16ce46b1d08047c28878bebcb0d58f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32843 } }
I20260923 15:10:38.590399 28981 leader_election.cc:302] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16ce46b1d08047c28878bebcb0d58f52; no voters:
I20260923 15:10:38.590595 28981 leader_election.cc:288] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:38.590616 28986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:38.590721 28986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [term 1 LEADER]: Becoming Leader. State: Replica: 16ce46b1d08047c28878bebcb0d58f52, State: Running, Role: LEADER
I20260923 15:10:38.590803 28986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16ce46b1d08047c28878bebcb0d58f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32843 } }
I20260923 15:10:38.590843 28981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:38.590903 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:38.592692 28988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16ce46b1d08047c28878bebcb0d58f52. Latest consensus state: current_term: 1 leader_uuid: "16ce46b1d08047c28878bebcb0d58f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16ce46b1d08047c28878bebcb0d58f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32843 } } }
I20260923 15:10:38.592962 28988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:38.592741 28987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16ce46b1d08047c28878bebcb0d58f52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16ce46b1d08047c28878bebcb0d58f52" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32843 } } }
I20260923 15:10:38.593148 28987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52 [sys.catalog]: This master's current role is: LEADER
W20260923 15:10:38.593286 29005 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:10:38.593343 29005 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:38.593453 29006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:38.593647 29006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:38.595292 29006 catalog_manager.cc:1347] Generated new cluster ID: a3ef11ef885c45f6979a4fe95c1cb02b
I20260923 15:10:38.595379 29006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:38.596282 28984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:38.592808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35429 (local address 127.0.0.1:32843)
0923 15:10:38.592844 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:38.592846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:38.592854 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:10:38.592874 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:38.592875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:38.592883 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:38.592938 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:38.593295 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:38.593956 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:38.594944 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:38.595224 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:38.595295 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:38.596045 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:38.596047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:38.596047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:38.596052 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:38.596067 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:38.596072 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:38.596133 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:38.596187 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 15:10:38.596211 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:10:38.613775 29006 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:38.614454 29006 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:38.626324 29006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16ce46b1d08047c28878bebcb0d58f52: Generated new TSK 0
I20260923 15:10:38.626566 29006 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:38.629232 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 28912
2026-09-23T15:10:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:38 dist-test-slave-bb3x krb5kdc[29012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:38 dist-test-slave-bb3x krb5kdc[29012](info): set up 2 sockets
Sep 23 15:10:38 dist-test-slave-bb3x krb5kdc[29012](info): commencing operation
krb5kdc: starting...
W20260923 15:10:39.703896 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:39 dist-test-slave-bb3x krb5kdc[29012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:39.831629 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34197
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:39.916440 29025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:39.916690 29025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:39.916729 29025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:39.917932 29025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:39.918001 29025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:39.918027 29025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:39.918061 29025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:39.918093 29025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:39.919694 29025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34197
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:39.919981 29025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:39.920223 29025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:39.925788 29033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:39.925798 29031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:39.925798 29030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:39.926127 29025 server_base.cc:1034] running on GCE node
I20260923 15:10:39.926398 29025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:39.926692 29025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:39.927860 29025 hybrid_clock.cc:648] HybridClock initialized: now 1790176239927846 us; error 29 us; skew 500 ppm
Sep 23 15:10:39 dist-test-slave-bb3x krb5kdc[29012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176239, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:39.930771 29025 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:39.931300 29025 webserver.cc:460] Webserver started at http://127.0.0.1:35877/ using document root <none> and password file <none>
I20260923 15:10:39.931634 29025 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:39.931684 29025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:39.931839 29025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:39.933017 29025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "e498b2d993bb44efa9f209996b9bde82"
format_stamp: "Formatted at 2026-09-23 15:10:39 on dist-test-slave-bb3x"
server_key: "84dff05673a9503ee7cd061db8eceb7a"
server_key_iv: "91ab420f9c3726167b4bbafc0fa825f1"
server_key_version: "encryptionkey@0"
I20260923 15:10:39.933388 29025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "e498b2d993bb44efa9f209996b9bde82"
format_stamp: "Formatted at 2026-09-23 15:10:39 on dist-test-slave-bb3x"
server_key: "84dff05673a9503ee7cd061db8eceb7a"
server_key_iv: "91ab420f9c3726167b4bbafc0fa825f1"
server_key_version: "encryptionkey@0"
I20260923 15:10:39.934839 29025 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:39.935806 29040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:39.936055 29025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:39.936134 29025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "e498b2d993bb44efa9f209996b9bde82"
format_stamp: "Formatted at 2026-09-23 15:10:39 on dist-test-slave-bb3x"
server_key: "84dff05673a9503ee7cd061db8eceb7a"
server_key_iv: "91ab420f9c3726167b4bbafc0fa825f1"
server_key_version: "encryptionkey@0"
I20260923 15:10:39.936230 29025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:39.939704 29025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:39.941912 29025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:39.942090 29025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:39.946489 29025 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34197
I20260923 15:10:39.946592 29092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34197 every 8 connection(s)
I20260923 15:10:39.946924 29025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:39.948100 29093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:39.950434 29093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82: Bootstrap starting.
I20260923 15:10:39.951030 29093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:39.951319 29093 log.cc:824] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:39.952165 29093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82: No bootstrap required, opened a new log
I20260923 15:10:39.953935 29093 raft_consensus.cc:348] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e498b2d993bb44efa9f209996b9bde82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260923 15:10:39.954105 29093 raft_consensus.cc:374] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:39.954154 29093 raft_consensus.cc:729] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e498b2d993bb44efa9f209996b9bde82, State: Initialized, Role: FOLLOWER
I20260923 15:10:39.954355 29093 consensus_queue.cc:260] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e498b2d993bb44efa9f209996b9bde82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260923 15:10:39.954464 29093 raft_consensus.cc:388] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:39.954509 29093 raft_consensus.cc:482] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:39.954558 29093 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:39.955106 29093 raft_consensus.cc:504] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e498b2d993bb44efa9f209996b9bde82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260923 15:10:39.955228 29093 leader_election.cc:302] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e498b2d993bb44efa9f209996b9bde82; no voters:
I20260923 15:10:39.955427 29093 leader_election.cc:288] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:39.955515 29096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:39.955715 29093 sys_catalog.cc:564] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:39.955725 29096 raft_consensus.cc:686] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [term 1 LEADER]: Becoming Leader. State: Replica: e498b2d993bb44efa9f209996b9bde82, State: Running, Role: LEADER
I20260923 15:10:39.955893 29096 consensus_queue.cc:237] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e498b2d993bb44efa9f209996b9bde82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } }
I20260923 15:10:39.956669 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29025
I20260923 15:10:39.956835 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:39.956825 29097 sys_catalog.cc:455] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e498b2d993bb44efa9f209996b9bde82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e498b2d993bb44efa9f209996b9bde82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } } }
I20260923 15:10:39.956943 29097 sys_catalog.cc:458] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:39.957103 8169 external_mini_cluster.cc:1428] Setting key aef5da7c59837a14cde72c3792c6c150
I20260923 15:10:39.957363 29110 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:39.957172 29098 sys_catalog.cc:455] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e498b2d993bb44efa9f209996b9bde82. Latest consensus state: current_term: 1 leader_uuid: "e498b2d993bb44efa9f209996b9bde82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e498b2d993bb44efa9f209996b9bde82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34197 } } }
I20260923 15:10:39.957477 29098 sys_catalog.cc:458] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:39.958127 29110 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:39.959899 29110 catalog_manager.cc:1347] Generated new cluster ID: 917a092e8c42423ab23e4b30eaf2feee
I20260923 15:10:39.959961 29110 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:39.960090 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:39.957809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55165 (local address 127.0.0.1:34197)
0923 15:10:39.958118 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:39.958127 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:39.958159 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:10:39.958300 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:39.958308 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:39.958343 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:39.958787 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:39.959442 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:39.959445 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:39.959445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:39.959447 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:39.959494 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:39.959509 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:39.959805 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:39.959839 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 15:10:39.959882 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":76,"threads_started":1}
I20260923 15:10:39.960772 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:39.963362 29115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:39.962253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46987 (local address 127.0.0.1:34197)
0923 15:10:39.962320 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:39.962322 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:39.962424 (+ 102us) server_negotiation.cc:407] Connection header received
0923 15:10:39.962451 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:39.962452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:39.962458 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:39.962538 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:39.963133 (+ 595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:39.963135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:39.963135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:39.963140 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:39.963157 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:39.963162 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:39.963230 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:39.963278 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 15:10:39.963293 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:10:39.976526 29110 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:39.976750 29110 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:39.984939 29110 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e498b2d993bb44efa9f209996b9bde82: Generated new TSK 0
I20260923 15:10:39.985157 29110 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:39.992861 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29025
2026-09-23T15:10:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:40 dist-test-slave-bb3x krb5kdc[29124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:40 dist-test-slave-bb3x krb5kdc[29124](info): set up 2 sockets
Sep 23 15:10:40 dist-test-slave-bb3x krb5kdc[29124](info): commencing operation
krb5kdc: starting...
W20260923 15:10:41.068024 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:41 dist-test-slave-bb3x krb5kdc[29124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:41.197824 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:41.281414 29137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:41.281657 29137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:41.281684 29137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:41.282861 29137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:41.282902 29137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:41.282917 29137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:41.282931 29137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:41.282943 29137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:41.284571 29137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:41.284777 29137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:41.284957 29137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:41.289963 29143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:41.290045 29145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:41.290164 29142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:41.290158 29137 server_base.cc:1034] running on GCE node
I20260923 15:10:41.290608 29137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:41.290973 29137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:41.292158 29137 hybrid_clock.cc:648] HybridClock initialized: now 1790176241292156 us; error 45 us; skew 500 ppm
Sep 23 15:10:41 dist-test-slave-bb3x krb5kdc[29124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176241, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:41.299288 29137 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:41.299779 29137 webserver.cc:460] Webserver started at http://127.0.0.1:38181/ using document root <none> and password file <none>
I20260923 15:10:41.300028 29137 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:41.300101 29137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:41.300249 29137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:41.301342 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "bcd8f8b54bfa47bdb81f545221aa674f"
format_stamp: "Formatted at 2026-09-23 15:10:41 on dist-test-slave-bb3x"
server_key: "fba717b6c00037a01a062cb4a58ec91e"
server_key_iv: "c473dc220585c535ccded8c486c6690f"
server_key_version: "encryptionkey@0"
I20260923 15:10:41.301827 29137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "bcd8f8b54bfa47bdb81f545221aa674f"
format_stamp: "Formatted at 2026-09-23 15:10:41 on dist-test-slave-bb3x"
server_key: "fba717b6c00037a01a062cb4a58ec91e"
server_key_iv: "c473dc220585c535ccded8c486c6690f"
server_key_version: "encryptionkey@0"
I20260923 15:10:41.303470 29137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:41.304384 29152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:41.304786 29137 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:41.304930 29137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "bcd8f8b54bfa47bdb81f545221aa674f"
format_stamp: "Formatted at 2026-09-23 15:10:41 on dist-test-slave-bb3x"
server_key: "fba717b6c00037a01a062cb4a58ec91e"
server_key_iv: "c473dc220585c535ccded8c486c6690f"
server_key_version: "encryptionkey@0"
I20260923 15:10:41.305017 29137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:41.328776 29137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:41.330847 29137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:41.331020 29137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:41.335107 29137 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42895
I20260923 15:10:41.335163 29204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42895 every 8 connection(s)
I20260923 15:10:41.335546 29137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:41.336530 29205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:41.338941 29205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f: Bootstrap starting.
I20260923 15:10:41.339530 29205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:41.339818 29205 log.cc:824] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:41.340517 29205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f: No bootstrap required, opened a new log
I20260923 15:10:41.342132 29205 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42895 } }
I20260923 15:10:41.342350 29205 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:41.342397 29205 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcd8f8b54bfa47bdb81f545221aa674f, State: Initialized, Role: FOLLOWER
I20260923 15:10:41.342549 29205 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42895 } }
I20260923 15:10:41.342624 29205 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:41.342646 29205 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:41.342716 29205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:41.343267 29205 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42895 } }
I20260923 15:10:41.343349 29205 leader_election.cc:302] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcd8f8b54bfa47bdb81f545221aa674f; no voters:
I20260923 15:10:41.343494 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29137
I20260923 15:10:41.343556 29205 leader_election.cc:288] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:41.343586 29208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:41.343642 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:41.343755 29208 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [term 1 LEADER]: Becoming Leader. State: Replica: bcd8f8b54bfa47bdb81f545221aa674f, State: Running, Role: LEADER
I20260923 15:10:41.343812 29205 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:41.343832 29208 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42895 } }
I20260923 15:10:41.343887 8169 external_mini_cluster.cc:1428] Setting key d18d3d9cea2a1d8a302c069e8fa4e334
I20260923 15:10:41.344116 29208 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcd8f8b54bfa47bdb81f545221aa674f. Latest consensus state: current_term: 1 leader_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42895 } } }
I20260923 15:10:41.344416 29210 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcd8f8b54bfa47bdb81f545221aa674f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42895 } } }
I20260923 15:10:41.344573 29208 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:41.344826 29210 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:41.345521 29220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:41.346081 29220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:41.347887 29220 catalog_manager.cc:1347] Generated new cluster ID: 5653a4e2b0e747c68720ae11d157dbe1
I20260923 15:10:41.347941 29220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:10:41 dist-test-slave-bb3x krb5kdc[29124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176241, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:10:41.354132 29220 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:41.354856 29220 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:41.355154 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:41.344333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:56041 (local address 127.0.0.1:42895)
0923 15:10:41.344675 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:41.344684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:41.344721 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:10:41.344864 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:41.344870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:41.344930 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:41.345121 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:41.345918 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:41.347332 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:41.348300 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:41.348714 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:41.348834 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:41.351244 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:41.351247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:41.351248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:41.351260 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:41.353425 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:41.353913 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:41.353917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:41.353917 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:41.354035 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:41.354361 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:41.354362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:41.354363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:41.354590 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:41.354784 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:41.354871 (+ 87us) server_negotiation.cc:299] Negotiation successful
0923 15:10:41.354995 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":93,"threads_started":1}
I20260923 15:10:41.356141 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:41.366582 29220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcd8f8b54bfa47bdb81f545221aa674f: Generated new TSK 0
I20260923 15:10:41.366807 29220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:41.372704 29217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:41.357521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:48869 (local address 127.0.0.1:42895)
0923 15:10:41.357581 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:41.357583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:41.366162 (+ 8579us) server_negotiation.cc:407] Connection header received
0923 15:10:41.366293 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:41.366294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:41.366308 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:41.366381 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:41.367123 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:41.367647 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:41.368423 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:41.368676 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:41.368764 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:41.370008 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:41.370010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:41.370010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:41.370018 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:41.371508 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:41.371887 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:41.371889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:41.371889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:41.371951 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:41.372178 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:41.372179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:41.372179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:41.372299 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:41.372421 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:41.372532 (+ 111us) server_negotiation.cc:299] Negotiation successful
0923 15:10:41.372598 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:10:41.374645 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29137
W20260923 15:10:41.379190 29122 connection.cc:535] client connection to 127.0.0.1:42895 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42895): Connection reset by peer (error 104)
2026-09-23T15:10:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:41 dist-test-slave-bb3x krb5kdc[29236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:41 dist-test-slave-bb3x krb5kdc[29236](info): set up 2 sockets
Sep 23 15:10:41 dist-test-slave-bb3x krb5kdc[29236](info): commencing operation
krb5kdc: starting...
W20260923 15:10:42.411968 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:42 dist-test-slave-bb3x krb5kdc[29236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:42.537775 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40807
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:42.628515 29249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:42.628738 29249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:42.628764 29249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:42.629971 29249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:42.630012 29249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:42.630029 29249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:42.630043 29249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:42.630056 29249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:42.631657 29249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:42.631870 29249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:42.632052 29249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:42.637727 29254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:42.637780 29249 server_base.cc:1034] running on GCE node
W20260923 15:10:42.637744 29255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:42.637748 29257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:42.638245 29249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:42.638639 29249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:42.639822 29249 hybrid_clock.cc:648] HybridClock initialized: now 1790176242639774 us; error 68 us; skew 500 ppm
Sep 23 15:10:42 dist-test-slave-bb3x krb5kdc[29236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176242, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:42.642494 29249 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:42.642943 29249 webserver.cc:460] Webserver started at http://127.0.0.1:43405/ using document root <none> and password file <none>
I20260923 15:10:42.643181 29249 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:42.643249 29249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:42.643396 29249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:42.644496 29249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "ad68281fdea74d72a3ac5ac6e46f55aa"
format_stamp: "Formatted at 2026-09-23 15:10:42 on dist-test-slave-bb3x"
server_key: "066983cc9c3a772e489366e2324e7fc7"
server_key_iv: "be51ed488a22cbb68247fc7985c3a615"
server_key_version: "encryptionkey@0"
I20260923 15:10:42.644881 29249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "ad68281fdea74d72a3ac5ac6e46f55aa"
format_stamp: "Formatted at 2026-09-23 15:10:42 on dist-test-slave-bb3x"
server_key: "066983cc9c3a772e489366e2324e7fc7"
server_key_iv: "be51ed488a22cbb68247fc7985c3a615"
server_key_version: "encryptionkey@0"
I20260923 15:10:42.646364 29249 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:10:42.647244 29264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:42.647455 29249 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:42.647584 29249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "ad68281fdea74d72a3ac5ac6e46f55aa"
format_stamp: "Formatted at 2026-09-23 15:10:42 on dist-test-slave-bb3x"
server_key: "066983cc9c3a772e489366e2324e7fc7"
server_key_iv: "be51ed488a22cbb68247fc7985c3a615"
server_key_version: "encryptionkey@0"
I20260923 15:10:42.647676 29249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:42.661798 29249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:42.664211 29249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:42.664386 29249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:42.668432 29249 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40807
I20260923 15:10:42.668478 29316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40807 every 8 connection(s)
I20260923 15:10:42.668859 29249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:42.669976 29317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:42.672343 29317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa: Bootstrap starting.
I20260923 15:10:42.672912 29317 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:42.673161 29317 log.cc:824] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:42.673259 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29249
I20260923 15:10:42.673463 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:42.673672 8169 external_mini_cluster.cc:1428] Setting key 2c43a9e6b6105d0462b94cc8186455ed
I20260923 15:10:42.673993 29317 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa: No bootstrap required, opened a new log
I20260923 15:10:42.675757 29317 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260923 15:10:42.675896 29317 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:42.675920 29317 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad68281fdea74d72a3ac5ac6e46f55aa, State: Initialized, Role: FOLLOWER
I20260923 15:10:42.676088 29317 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260923 15:10:42.676156 29317 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:42.676178 29317 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:42.676246 29317 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:42.676822 29317 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260923 15:10:42.676941 29317 leader_election.cc:302] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad68281fdea74d72a3ac5ac6e46f55aa; no voters:
I20260923 15:10:42.677134 29317 leader_election.cc:288] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:42.677171 29322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:42.677323 29322 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [term 1 LEADER]: Becoming Leader. State: Replica: ad68281fdea74d72a3ac5ac6e46f55aa, State: Running, Role: LEADER
I20260923 15:10:42.677434 29317 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:42.677456 29322 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } }
I20260923 15:10:42.677853 29324 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad68281fdea74d72a3ac5ac6e46f55aa. Latest consensus state: current_term: 1 leader_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } } }
I20260923 15:10:42.678104 29324 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:42.678012 29323 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad68281fdea74d72a3ac5ac6e46f55aa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40807 } } }
I20260923 15:10:42.678241 29323 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:42.678539 29326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:42.679105 29326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:42.681191 29326 catalog_manager.cc:1347] Generated new cluster ID: 55cb10504424430486fd6bc762172e1f
I20260923 15:10:42.681254 29326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:42.681697 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:42.674194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:60807 (local address 127.0.0.1:40807)
0923 15:10:42.674392 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:42.674404 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:42.674465 (+ 61us) server_negotiation.cc:407] Connection header received
0923 15:10:42.675771 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:42.675780 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:42.675838 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:42.676027 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:42.676380 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:42.677825 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:42.679072 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:42.679811 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:42.679950 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:42.680723 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:42.680727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:42.680727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:42.680984 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:42.681031 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:42.681047 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:42.681418 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:42.681464 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 15:10:42.681492 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260923 15:10:42.682693 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:42.688086 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:42.684253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:44025 (local address 127.0.0.1:40807)
0923 15:10:42.684308 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:42.684310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:42.684393 (+ 83us) server_negotiation.cc:407] Connection header received
0923 15:10:42.684453 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:42.684454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:42.684466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:42.684518 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:42.684651 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:42.685401 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:42.686564 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:42.686895 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:42.686940 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:42.687760 (+ 820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:42.687762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:42.687762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:42.687767 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:42.687783 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:42.687788 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:42.687844 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:42.687983 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 15:10:42.687998 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:10:42.701965 29326 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:42.702600 29326 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:42.707350 29326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad68281fdea74d72a3ac5ac6e46f55aa: Generated new TSK 0
I20260923 15:10:42.707544 29326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:42.709939 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29249
W20260923 15:10:42.716090 29234 connection.cc:535] client connection to 127.0.0.1:40807 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40807): Connection reset by peer (error 104)
2026-09-23T15:10:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:42 dist-test-slave-bb3x krb5kdc[29348](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:42 dist-test-slave-bb3x krb5kdc[29348](info): set up 2 sockets
Sep 23 15:10:42 dist-test-slave-bb3x krb5kdc[29348](info): commencing operation
krb5kdc: starting...
W20260923 15:10:43.816423 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:43 dist-test-slave-bb3x krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:43.946583 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:44.028905 29361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:44.029165 29361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:44.029204 29361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:44.030464 29361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:44.030546 29361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:44.030572 29361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:44.030601 29361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:44.030633 29361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:44.032105 29361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:44.032390 29361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:44.032617 29361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:44.037786 29369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:44.037870 29367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:44.037958 29361 server_base.cc:1034] running on GCE node
W20260923 15:10:44.037813 29366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:44.038379 29361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:44.038728 29361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:44.039938 29361 hybrid_clock.cc:648] HybridClock initialized: now 1790176244039925 us; error 67 us; skew 500 ppm
Sep 23 15:10:44 dist-test-slave-bb3x krb5kdc[29348](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176244, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:44.043411 29361 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:44.043877 29361 webserver.cc:460] Webserver started at http://127.0.0.1:39263/ using document root <none> and password file <none>
I20260923 15:10:44.044112 29361 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:44.044157 29361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:44.044324 29361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:44.045460 29361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "4d1f50b9196b4476b03702eef8f5c073"
format_stamp: "Formatted at 2026-09-23 15:10:44 on dist-test-slave-bb3x"
server_key: "8d5463d751aaee441c39a262328911c2"
server_key_iv: "8ee720267fa5393e47749c23e7016c57"
server_key_version: "encryptionkey@0"
I20260923 15:10:44.045835 29361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "4d1f50b9196b4476b03702eef8f5c073"
format_stamp: "Formatted at 2026-09-23 15:10:44 on dist-test-slave-bb3x"
server_key: "8d5463d751aaee441c39a262328911c2"
server_key_iv: "8ee720267fa5393e47749c23e7016c57"
server_key_version: "encryptionkey@0"
I20260923 15:10:44.047272 29361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:10:44.048089 29376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:44.048282 29361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:10:44.048420 29361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "4d1f50b9196b4476b03702eef8f5c073"
format_stamp: "Formatted at 2026-09-23 15:10:44 on dist-test-slave-bb3x"
server_key: "8d5463d751aaee441c39a262328911c2"
server_key_iv: "8ee720267fa5393e47749c23e7016c57"
server_key_version: "encryptionkey@0"
I20260923 15:10:44.048491 29361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:44.051875 29361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:44.053841 29361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:44.053952 29361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:44.057775 29361 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43295
I20260923 15:10:44.057960 29428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43295 every 8 connection(s)
I20260923 15:10:44.058176 29361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:44.059181 29429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:44.061340 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29361
I20260923 15:10:44.061452 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:44.061595 29429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073: Bootstrap starting.
I20260923 15:10:44.061724 8169 external_mini_cluster.cc:1428] Setting key a77e49fd7b80c46e3613884818a33be8
I20260923 15:10:44.062420 29429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:44.062775 29429 log.cc:824] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:44.063555 29429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073: No bootstrap required, opened a new log
I20260923 15:10:44.064762 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:44.062445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:36743 (local address 127.0.0.1:43295)
0923 15:10:44.062683 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:44.062690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:44.062727 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:10:44.063242 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:44.063249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:44.063277 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:44.063398 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:44.064149 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:44.064152 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:44.064153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:44.064157 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:44.064201 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:44.064212 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:44.064377 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:44.064454 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 15:10:44.064534 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260923 15:10:44.065989 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:44.066030 29429 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1f50b9196b4476b03702eef8f5c073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260923 15:10:44.066210 29429 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:44.066252 29429 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d1f50b9196b4476b03702eef8f5c073, State: Initialized, Role: FOLLOWER
I20260923 15:10:44.066411 29429 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1f50b9196b4476b03702eef8f5c073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260923 15:10:44.066502 29429 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:44.066526 29429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:44.066594 29429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:44.067163 29429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1f50b9196b4476b03702eef8f5c073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260923 15:10:44.067276 29429 leader_election.cc:302] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d1f50b9196b4476b03702eef8f5c073; no voters:
I20260923 15:10:44.067476 29429 leader_election.cc:288] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:44.067499 29438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:44.067735 29429 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:44.067888 29438 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [term 1 LEADER]: Becoming Leader. State: Replica: 4d1f50b9196b4476b03702eef8f5c073, State: Running, Role: LEADER
I20260923 15:10:44.067994 29438 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1f50b9196b4476b03702eef8f5c073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } }
I20260923 15:10:44.068322 29438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d1f50b9196b4476b03702eef8f5c073. Latest consensus state: current_term: 1 leader_uuid: "4d1f50b9196b4476b03702eef8f5c073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1f50b9196b4476b03702eef8f5c073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } } }
I20260923 15:10:44.068347 29441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d1f50b9196b4476b03702eef8f5c073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d1f50b9196b4476b03702eef8f5c073" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43295 } } }
I20260923 15:10:44.068456 29438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:44.068471 29441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:44.068691 29448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:44.069265 29448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:44.069924 29432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:44.067753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:41773 (local address 127.0.0.1:43295)
0923 15:10:44.067795 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:44.067797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:44.068753 (+ 956us) server_negotiation.cc:407] Connection header received
0923 15:10:44.068896 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:44.068896 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:44.068902 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:44.068937 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:44.069688 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:44.069689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:44.069689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:44.069694 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:44.069707 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:44.069711 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:44.069740 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:44.069843 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 15:10:44.069867 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:10:44.075453 29448 catalog_manager.cc:1347] Generated new cluster ID: bfeac25612ba4a8f8e3352afdb92c0b1
I20260923 15:10:44.075518 29448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:44.095180 29448 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:44.095476 29448 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:44.102169 29448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d1f50b9196b4476b03702eef8f5c073: Generated new TSK 0
I20260923 15:10:44.102514 29448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:44.111018 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29361
2026-09-23T15:10:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:44 dist-test-slave-bb3x krb5kdc[29460](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:44 dist-test-slave-bb3x krb5kdc[29460](info): set up 2 sockets
Sep 23 15:10:44 dist-test-slave-bb3x krb5kdc[29460](info): commencing operation
krb5kdc: starting...
W20260923 15:10:45.155988 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:45 dist-test-slave-bb3x krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:45.282171 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:45.363438 29473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:45.363637 29473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:45.363660 29473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:45.364817 29473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:45.364861 29473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:45.364874 29473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:45.364887 29473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:45.364900 29473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:45.366403 29473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:45.366657 29473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:45.366894 29473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:45.372040 29478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:45.372016 29479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:45.372040 29481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:45.372360 29473 server_base.cc:1034] running on GCE node
I20260923 15:10:45.372606 29473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:45.372900 29473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:45.374100 29473 hybrid_clock.cc:648] HybridClock initialized: now 1790176245374083 us; error 33 us; skew 500 ppm
Sep 23 15:10:45 dist-test-slave-bb3x krb5kdc[29460](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176245, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:45.376945 29473 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:45.377511 29473 webserver.cc:460] Webserver started at http://127.0.0.1:40895/ using document root <none> and password file <none>
I20260923 15:10:45.377768 29473 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:45.377839 29473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:45.377990 29473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:45.379135 29473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "72c78f3879ad4a3cbdd262e19358e14e"
format_stamp: "Formatted at 2026-09-23 15:10:45 on dist-test-slave-bb3x"
server_key: "94c949dc03d7c40612b9d406c3ec30a0"
server_key_iv: "0b1bb8e0cd7f7a669c80c9b3bd057fbd"
server_key_version: "encryptionkey@0"
I20260923 15:10:45.379513 29473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "72c78f3879ad4a3cbdd262e19358e14e"
format_stamp: "Formatted at 2026-09-23 15:10:45 on dist-test-slave-bb3x"
server_key: "94c949dc03d7c40612b9d406c3ec30a0"
server_key_iv: "0b1bb8e0cd7f7a669c80c9b3bd057fbd"
server_key_version: "encryptionkey@0"
I20260923 15:10:45.380952 29473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:10:45.381764 29488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:45.382000 29473 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:10:45.382121 29473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "72c78f3879ad4a3cbdd262e19358e14e"
format_stamp: "Formatted at 2026-09-23 15:10:45 on dist-test-slave-bb3x"
server_key: "94c949dc03d7c40612b9d406c3ec30a0"
server_key_iv: "0b1bb8e0cd7f7a669c80c9b3bd057fbd"
server_key_version: "encryptionkey@0"
I20260923 15:10:45.382201 29473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:45.393980 29473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:45.396494 29473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:45.396677 29473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:45.401008 29540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41961 every 8 connection(s)
I20260923 15:10:45.400975 29473 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41961
I20260923 15:10:45.401685 29473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:45.402769 29541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:45.405136 29541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e: Bootstrap starting.
I20260923 15:10:45.405745 29541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:45.406045 29541 log.cc:824] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:45.406919 29541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e: No bootstrap required, opened a new log
I20260923 15:10:45.407509 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29473
I20260923 15:10:45.407603 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:45.407819 8169 external_mini_cluster.cc:1428] Setting key bee363f629fdee2c3893fe2ce9c61a8a
I20260923 15:10:45.409709 29541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c78f3879ad4a3cbdd262e19358e14e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260923 15:10:45.409880 29541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:45.409930 29541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72c78f3879ad4a3cbdd262e19358e14e, State: Initialized, Role: FOLLOWER
I20260923 15:10:45.410077 29541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c78f3879ad4a3cbdd262e19358e14e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260923 15:10:45.410173 29541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:45.410220 29541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:45.410298 29541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:45.411175 29541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c78f3879ad4a3cbdd262e19358e14e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260923 15:10:45.411301 29541 leader_election.cc:302] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72c78f3879ad4a3cbdd262e19358e14e; no voters:
I20260923 15:10:45.411492 29541 leader_election.cc:288] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:45.411563 29546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:45.411746 29546 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [term 1 LEADER]: Becoming Leader. State: Replica: 72c78f3879ad4a3cbdd262e19358e14e, State: Running, Role: LEADER
I20260923 15:10:45.411760 29541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:45.411906 29546 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c78f3879ad4a3cbdd262e19358e14e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260923 15:10:45.412250 29546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72c78f3879ad4a3cbdd262e19358e14e. Latest consensus state: current_term: 1 leader_uuid: "72c78f3879ad4a3cbdd262e19358e14e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c78f3879ad4a3cbdd262e19358e14e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } } }
I20260923 15:10:45.412364 29546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:45.412261 29547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72c78f3879ad4a3cbdd262e19358e14e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72c78f3879ad4a3cbdd262e19358e14e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } } }
I20260923 15:10:45.412493 29547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:45.413260 29551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:45.413841 29551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:45.415673 29551 catalog_manager.cc:1347] Generated new cluster ID: 6447f29bca394161b0d0d2aa0ae3aead
I20260923 15:10:45.415884 29551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:10:45 dist-test-slave-bb3x krb5kdc[29460](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:10:45.422149 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:45.408408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:36727 (local address 127.0.0.1:41961)
0923 15:10:45.408620 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:45.408631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:45.408668 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:10:45.409241 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:45.409251 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:45.409323 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:45.409497 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:45.410184 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:45.412147 (+ 1963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:45.413554 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:45.414242 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:45.414395 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:45.417994 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:45.417999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:45.418000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:45.418014 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:45.420175 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:45.420695 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:45.420699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:45.420699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:45.420816 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:45.421077 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:45.421078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:45.421078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:45.421395 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:45.421540 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:45.421802 (+ 262us) server_negotiation.cc:299] Negotiation successful
0923 15:10:45.421920 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":99,"threads_started":1}
I20260923 15:10:45.423285 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:45.432670 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:45.424850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:51707 (local address 127.0.0.1:41961)
0923 15:10:45.424891 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:45.424893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:45.424939 (+ 46us) server_negotiation.cc:407] Connection header received
0923 15:10:45.425016 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:45.425017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:45.425028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:45.425086 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:45.425745 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:45.426621 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:45.427696 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:45.428041 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:45.428113 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:45.429560 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:45.429562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:45.429562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:45.429571 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:45.431276 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:45.431722 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:45.431723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:45.431723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:45.431784 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:45.432020 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:45.432020 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:45.432021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:45.432152 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:45.432259 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:45.432401 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 15:10:45.432464 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:10:45.447227 29551 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:45.447829 29551 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:45.455718 29551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72c78f3879ad4a3cbdd262e19358e14e: Generated new TSK 0
I20260923 15:10:45.455960 29551 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:45.460515 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29473
W20260923 15:10:45.464834 29458 connection.cc:535] client connection to 127.0.0.1:41961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41961): Connection reset by peer (error 104)
2026-09-23T15:10:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:45 dist-test-slave-bb3x krb5kdc[29572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:45 dist-test-slave-bb3x krb5kdc[29572](info): set up 2 sockets
Sep 23 15:10:45 dist-test-slave-bb3x krb5kdc[29572](info): commencing operation
krb5kdc: starting...
W20260923 15:10:46.592127 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:46 dist-test-slave-bb3x krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:46.722920 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34805
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:46.805491 29585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:46.805707 29585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:46.805732 29585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:46.806885 29585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:46.806922 29585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:46.806937 29585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:46.806949 29585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:46.806962 29585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:46.808379 29585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34805
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:46.808573 29585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:46.808740 29585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:46.814114 29591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:46.814147 29593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:46.814121 29590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:46.814110 29585 server_base.cc:1034] running on GCE node
I20260923 15:10:46.814618 29585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:46.814949 29585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:46.816094 29585 hybrid_clock.cc:648] HybridClock initialized: now 1790176246816072 us; error 34 us; skew 500 ppm
Sep 23 15:10:46 dist-test-slave-bb3x krb5kdc[29572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176246, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:46.824280 29585 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:46.824760 29585 webserver.cc:460] Webserver started at http://127.0.0.1:46087/ using document root <none> and password file <none>
I20260923 15:10:46.824962 29585 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:46.825006 29585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:46.825094 29585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:46.826109 29585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "c2982663f54441c6ac529e87f0642abd"
format_stamp: "Formatted at 2026-09-23 15:10:46 on dist-test-slave-bb3x"
server_key: "794706211fcf09be1bc034a651ea55ee"
server_key_iv: "490a581ee5f6bcf854d32da62b2e1bae"
server_key_version: "encryptionkey@0"
I20260923 15:10:46.826483 29585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "c2982663f54441c6ac529e87f0642abd"
format_stamp: "Formatted at 2026-09-23 15:10:46 on dist-test-slave-bb3x"
server_key: "794706211fcf09be1bc034a651ea55ee"
server_key_iv: "490a581ee5f6bcf854d32da62b2e1bae"
server_key_version: "encryptionkey@0"
I20260923 15:10:46.827787 29585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:46.828503 29600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:46.828693 29585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:10:46.828807 29585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "c2982663f54441c6ac529e87f0642abd"
format_stamp: "Formatted at 2026-09-23 15:10:46 on dist-test-slave-bb3x"
server_key: "794706211fcf09be1bc034a651ea55ee"
server_key_iv: "490a581ee5f6bcf854d32da62b2e1bae"
server_key_version: "encryptionkey@0"
I20260923 15:10:46.828902 29585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:46.849064 29585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:46.851500 29585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:46.851672 29585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:46.855821 29585 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34805
I20260923 15:10:46.855860 29652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34805 every 8 connection(s)
I20260923 15:10:46.856259 29585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:46.857138 29653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:46.858377 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29585
I20260923 15:10:46.858486 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:46.858706 8169 external_mini_cluster.cc:1428] Setting key 536d2c0b35e5239431ea1e8c7bc07fc4
I20260923 15:10:46.860278 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd: Bootstrap starting.
I20260923 15:10:46.860867 29653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:46.861163 29653 log.cc:824] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:46.861861 29653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd: No bootstrap required, opened a new log
I20260923 15:10:46.863783 29653 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2982663f54441c6ac529e87f0642abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260923 15:10:46.863937 29653 raft_consensus.cc:374] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:46.863984 29653 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2982663f54441c6ac529e87f0642abd, State: Initialized, Role: FOLLOWER
I20260923 15:10:46.864120 29653 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2982663f54441c6ac529e87f0642abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260923 15:10:46.864230 29653 raft_consensus.cc:388] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:46.864320 29653 raft_consensus.cc:482] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:46.864408 29653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:46.864511 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:46.859242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:48743 (local address 127.0.0.1:34805)
0923 15:10:46.859446 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:46.859455 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:46.859488 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:10:46.860010 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:46.860019 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:46.860082 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:46.860232 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:46.860469 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:46.861589 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:46.862502 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:46.862918 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:46.863048 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:46.863751 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:46.863755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:46.863755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:46.863933 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:46.863984 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:46.864000 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:46.864082 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:46.864242 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 15:10:46.864273 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":89,"threads_started":1}
I20260923 15:10:46.865135 29653 raft_consensus.cc:504] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2982663f54441c6ac529e87f0642abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260923 15:10:46.865273 29653 leader_election.cc:302] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2982663f54441c6ac529e87f0642abd; no voters:
I20260923 15:10:46.865437 29653 leader_election.cc:288] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:46.865456 29658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:46.865513 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:46.865640 29658 raft_consensus.cc:686] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [term 1 LEADER]: Becoming Leader. State: Replica: c2982663f54441c6ac529e87f0642abd, State: Running, Role: LEADER
I20260923 15:10:46.865710 29653 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:46.865717 29658 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2982663f54441c6ac529e87f0642abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } }
I20260923 15:10:46.866025 29658 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2982663f54441c6ac529e87f0642abd. Latest consensus state: current_term: 1 leader_uuid: "c2982663f54441c6ac529e87f0642abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2982663f54441c6ac529e87f0642abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } } }
I20260923 15:10:46.866101 29658 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:46.866037 29659 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2982663f54441c6ac529e87f0642abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2982663f54441c6ac529e87f0642abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34805 } } }
I20260923 15:10:46.866146 29659 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:46.866321 29664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:46.866846 29664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:46.868546 29664 catalog_manager.cc:1347] Generated new cluster ID: 3dc00de357794137bca7a6222006d0b4
I20260923 15:10:46.868598 29664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:46.871042 29656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:46.867302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:33555 (local address 127.0.0.1:34805)
0923 15:10:46.867336 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:46.867337 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:46.867594 (+ 257us) server_negotiation.cc:407] Connection header received
0923 15:10:46.867686 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:46.867686 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:46.867696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:46.867724 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:46.867920 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:46.868633 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:46.869640 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:46.869982 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:46.870061 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:46.870750 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:46.870752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:46.870752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:46.870756 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:46.870771 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:46.870776 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:46.870846 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:46.870953 (+ 107us) server_negotiation.cc:299] Negotiation successful
0923 15:10:46.870965 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:10:46.888207 29664 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:46.888803 29664 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:46.893008 29664 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c2982663f54441c6ac529e87f0642abd: Generated new TSK 0
I20260923 15:10:46.893247 29664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:46.903689 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29585
W20260923 15:10:46.908401 29570 connection.cc:535] client connection to 127.0.0.1:34805 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34805): Connection reset by peer (error 104)
2026-09-23T15:10:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:46 dist-test-slave-bb3x krb5kdc[29684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:46 dist-test-slave-bb3x krb5kdc[29684](info): set up 2 sockets
Sep 23 15:10:46 dist-test-slave-bb3x krb5kdc[29684](info): commencing operation
krb5kdc: starting...
W20260923 15:10:47.952013 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:48 dist-test-slave-bb3x krb5kdc[29684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176248, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:10:48.081852 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:48.163786 29697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:48.164062 29697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:48.164103 29697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:48.165580 29697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:48.165688 29697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:48.165715 29697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:48.165736 29697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:48.165769 29697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:48.167476 29697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39669
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:48.167791 29697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:48.168042 29697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:48.173141 29702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:48.173300 29697 server_base.cc:1034] running on GCE node
W20260923 15:10:48.173161 29703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:48.173377 29705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:48.173684 29697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:48.173954 29697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:48.175135 29697 hybrid_clock.cc:648] HybridClock initialized: now 1790176248175118 us; error 43 us; skew 500 ppm
Sep 23 15:10:48 dist-test-slave-bb3x krb5kdc[29684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176248, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:48.177879 29697 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:10:48.178385 29697 webserver.cc:460] Webserver started at http://127.0.0.1:40119/ using document root <none> and password file <none>
I20260923 15:10:48.178622 29697 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:48.178689 29697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:48.178840 29697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:48.179944 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "e785d25e06d34a1f97cdabff5c66594e"
format_stamp: "Formatted at 2026-09-23 15:10:48 on dist-test-slave-bb3x"
server_key: "ca42f1dc89e27703eebd5372315ce6f2"
server_key_iv: "d881b316b573e3f33bd0fb377f4f1404"
server_key_version: "encryptionkey@0"
I20260923 15:10:48.180331 29697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "e785d25e06d34a1f97cdabff5c66594e"
format_stamp: "Formatted at 2026-09-23 15:10:48 on dist-test-slave-bb3x"
server_key: "ca42f1dc89e27703eebd5372315ce6f2"
server_key_iv: "d881b316b573e3f33bd0fb377f4f1404"
server_key_version: "encryptionkey@0"
I20260923 15:10:48.181706 29697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:48.182607 29712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:48.182854 29697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:48.182965 29697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "e785d25e06d34a1f97cdabff5c66594e"
format_stamp: "Formatted at 2026-09-23 15:10:48 on dist-test-slave-bb3x"
server_key: "ca42f1dc89e27703eebd5372315ce6f2"
server_key_iv: "d881b316b573e3f33bd0fb377f4f1404"
server_key_version: "encryptionkey@0"
I20260923 15:10:48.183040 29697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:48.186468 29697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:48.188860 29697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:48.189030 29697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:48.193413 29697 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39669
I20260923 15:10:48.193449 29764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39669 every 8 connection(s)
I20260923 15:10:48.193810 29697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:48.194921 29765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:48.196942 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29697
I20260923 15:10:48.197081 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:48.197270 8169 external_mini_cluster.cc:1428] Setting key e068dbf6a3c85d29c49779581b76ccd8
I20260923 15:10:48.197494 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e: Bootstrap starting.
I20260923 15:10:48.198251 29765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:48.198596 29765 log.cc:824] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:48.199399 29765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e: No bootstrap required, opened a new log
I20260923 15:10:48.200241 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:48.197920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:45351 (local address 127.0.0.1:39669)
0923 15:10:48.198124 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:48.198132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:48.198159 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:10:48.198723 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:48.198732 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:48.198772 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:48.198924 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:48.199703 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:48.199706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:48.199706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:48.199711 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:48.199759 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:48.199769 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:48.199903 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:48.199943 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 15:10:48.199989 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":94,"threads_started":1}
I20260923 15:10:48.201612 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:48.202231 29765 raft_consensus.cc:348] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785d25e06d34a1f97cdabff5c66594e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260923 15:10:48.202476 29765 raft_consensus.cc:374] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:48.202526 29765 raft_consensus.cc:729] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e785d25e06d34a1f97cdabff5c66594e, State: Initialized, Role: FOLLOWER
I20260923 15:10:48.202663 29765 consensus_queue.cc:260] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785d25e06d34a1f97cdabff5c66594e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260923 15:10:48.202786 29765 raft_consensus.cc:388] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:48.202845 29765 raft_consensus.cc:482] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:48.202898 29765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:48.203783 29765 raft_consensus.cc:504] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785d25e06d34a1f97cdabff5c66594e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260923 15:10:48.203905 29765 leader_election.cc:302] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e785d25e06d34a1f97cdabff5c66594e; no voters:
I20260923 15:10:48.204079 29765 leader_election.cc:288] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:48.204128 29775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:48.204243 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:48.203121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.104:41187 (local address 127.0.0.1:39669)
0923 15:10:48.203199 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:48.203201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:48.203268 (+ 67us) server_negotiation.cc:407] Connection header received
0923 15:10:48.203343 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:48.203343 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:48.203347 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:48.203392 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:48.203973 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:48.203975 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:48.203975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:48.203980 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:48.203996 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:48.204000 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:48.204073 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:48.204117 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 15:10:48.204131 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 15:10:48.204391 29765 sys_catalog.cc:564] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:48.204326 29775 raft_consensus.cc:686] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [term 1 LEADER]: Becoming Leader. State: Replica: e785d25e06d34a1f97cdabff5c66594e, State: Running, Role: LEADER
I20260923 15:10:48.204824 29775 consensus_queue.cc:237] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785d25e06d34a1f97cdabff5c66594e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } }
I20260923 15:10:48.205231 29776 sys_catalog.cc:455] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e785d25e06d34a1f97cdabff5c66594e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785d25e06d34a1f97cdabff5c66594e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } } }
I20260923 15:10:48.205246 29777 sys_catalog.cc:455] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e785d25e06d34a1f97cdabff5c66594e. Latest consensus state: current_term: 1 leader_uuid: "e785d25e06d34a1f97cdabff5c66594e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e785d25e06d34a1f97cdabff5c66594e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39669 } } }
I20260923 15:10:48.205331 29776 sys_catalog.cc:458] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:48.205348 29777 sys_catalog.cc:458] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:48.205530 29785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:48.206032 29785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:48.207870 29785 catalog_manager.cc:1347] Generated new cluster ID: bdd4d13ce1c84970aa8dc0bb56a3bacb
I20260923 15:10:48.207921 29785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:48.221742 29785 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:48.221956 29785 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:48.228034 29785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e785d25e06d34a1f97cdabff5c66594e: Generated new TSK 0
I20260923 15:10:48.228363 29785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:48.235370 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29697
2026-09-23T15:10:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1330 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:48 dist-test-slave-bb3x krb5kdc[29796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:48 dist-test-slave-bb3x krb5kdc[29796](info): set up 2 sockets
Sep 23 15:10:48 dist-test-slave-bb3x krb5kdc[29796](info): commencing operation
krb5kdc: starting...
W20260923 15:10:49.275949 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:49 dist-test-slave-bb3x krb5kdc[29796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:49.401934 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44613
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:44613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:49.484172 29809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:49.484385 29809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:49.484411 29809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:49.485496 29809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:49.485530 29809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:49.485544 29809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:49.485556 29809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:49.485569 29809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:49.487003 29809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:44613
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:44613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:49.487203 29809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:49.487378 29809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:10:49.492661 29809 server_base.cc:1034] running on GCE node
W20260923 15:10:49.492579 29815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:49.492611 29814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:49.492597 29817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:49.493069 29809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:49.493368 29809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:49.494540 29809 hybrid_clock.cc:648] HybridClock initialized: now 1790176249494531 us; error 41 us; skew 500 ppm
Sep 23 15:10:49 dist-test-slave-bb3x krb5kdc[29796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176249, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:49.497092 29809 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:49.497558 29809 webserver.cc:460] Webserver started at http://127.7.250.126:33099/ using document root <none> and password file <none>
I20260923 15:10:49.497762 29809 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:49.497803 29809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:49.497891 29809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:49.498939 29809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "79354075bae945ecb1fe881a18addd2b"
format_stamp: "Formatted at 2026-09-23 15:10:49 on dist-test-slave-bb3x"
server_key: "00524b4c7d815e13d5eb2e3429ad857c"
server_key_iv: "83d2961a1779739a0ce646147ca7e4a0"
server_key_version: "encryptionkey@0"
I20260923 15:10:49.499377 29809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "79354075bae945ecb1fe881a18addd2b"
format_stamp: "Formatted at 2026-09-23 15:10:49 on dist-test-slave-bb3x"
server_key: "00524b4c7d815e13d5eb2e3429ad857c"
server_key_iv: "83d2961a1779739a0ce646147ca7e4a0"
server_key_version: "encryptionkey@0"
I20260923 15:10:49.500749 29809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:49.501596 29824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:49.501822 29809 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:49.501952 29809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "79354075bae945ecb1fe881a18addd2b"
format_stamp: "Formatted at 2026-09-23 15:10:49 on dist-test-slave-bb3x"
server_key: "00524b4c7d815e13d5eb2e3429ad857c"
server_key_iv: "83d2961a1779739a0ce646147ca7e4a0"
server_key_version: "encryptionkey@0"
I20260923 15:10:49.502033 29809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:49.509869 29809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:49.512326 29809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:49.512504 29809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:49.516819 29809 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:44613
I20260923 15:10:49.516871 29876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:44613 every 8 connection(s)
I20260923 15:10:49.517254 29809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:49.518348 29877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:49.520602 29877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b: Bootstrap starting.
I20260923 15:10:49.521202 29877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:49.521487 29877 log.cc:824] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:49.522203 29877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b: No bootstrap required, opened a new log
I20260923 15:10:49.523877 29877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79354075bae945ecb1fe881a18addd2b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44613 } }
I20260923 15:10:49.524011 29877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:49.524034 29877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79354075bae945ecb1fe881a18addd2b, State: Initialized, Role: FOLLOWER
I20260923 15:10:49.524180 29877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79354075bae945ecb1fe881a18addd2b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44613 } }
I20260923 15:10:49.524263 29877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:49.524287 29877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:49.524338 29877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:49.525187 29877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79354075bae945ecb1fe881a18addd2b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44613 } }
I20260923 15:10:49.525291 29877 leader_election.cc:302] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79354075bae945ecb1fe881a18addd2b; no voters:
I20260923 15:10:49.525465 29877 leader_election.cc:288] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:49.525511 29880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:49.525720 29880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [term 1 LEADER]: Becoming Leader. State: Replica: 79354075bae945ecb1fe881a18addd2b, State: Running, Role: LEADER
I20260923 15:10:49.525731 29877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:49.525822 29880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79354075bae945ecb1fe881a18addd2b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44613 } }
I20260923 15:10:49.526197 29881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79354075bae945ecb1fe881a18addd2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79354075bae945ecb1fe881a18addd2b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44613 } } }
I20260923 15:10:49.526441 29881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:49.526367 29882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79354075bae945ecb1fe881a18addd2b. Latest consensus state: current_term: 1 leader_uuid: "79354075bae945ecb1fe881a18addd2b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79354075bae945ecb1fe881a18addd2b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 44613 } } }
I20260923 15:10:49.526549 29882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:49.526952 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29809
I20260923 15:10:49.527069 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:49.527069 29889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:49.527284 8169 external_mini_cluster.cc:1428] Setting key 2a78616657ab7439ffc1041e0387af56
I20260923 15:10:49.527598 29889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:49.529765 29889 catalog_manager.cc:1347] Generated new cluster ID: d36f29467a484c34939922a082d96904
I20260923 15:10:49.529815 29889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:10:49 dist-test-slave-bb3x krb5kdc[29796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:49.537283 29889 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:49.537884 29889 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:49.539330 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:49.527930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54641 (local address 127.7.250.126:44613)
0923 15:10:49.528112 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:49.528121 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:49.528509 (+ 388us) server_negotiation.cc:407] Connection header received
0923 15:10:49.528629 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:49.528636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:49.528703 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:49.528847 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:49.529465 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:49.530890 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:49.531923 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:49.532560 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:49.532672 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:49.535521 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:49.535525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:49.535526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:49.535538 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:49.537573 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:49.538042 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:49.538045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:49.538046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:49.538144 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:49.538435 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:49.538436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:49.538437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:49.538718 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:49.538883 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:49.538988 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 15:10:49.539090 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260923 15:10:49.540226 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:49.542709 29889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79354075bae945ecb1fe881a18addd2b: Generated new TSK 0
I20260923 15:10:49.542894 29889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:49.548492 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:49.541620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42761 (local address 127.7.250.126:44613)
0923 15:10:49.541682 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:49.541684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:49.541919 (+ 235us) server_negotiation.cc:407] Connection header received
0923 15:10:49.541971 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:49.541972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:49.541982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:49.542043 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:49.542835 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:49.543504 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:49.544269 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:49.544500 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:49.544577 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:49.545787 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:49.545788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:49.545788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:49.545796 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:49.547228 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:49.547600 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:49.547601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:49.547602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:49.547682 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:49.547907 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:49.547909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:49.547909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:49.548016 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:49.548125 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:49.548330 (+ 205us) server_negotiation.cc:299] Negotiation successful
0923 15:10:49.548396 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":41}
I20260923 15:10:49.550571 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29809
W20260923 15:10:49.554579 29794 connection.cc:535] client connection to 127.7.250.126:44613 recv error: Network error: failed to read from TLS socket (remote: 127.7.250.126:44613): Connection reset by peer (error 104)
2026-09-23T15:10:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:49 dist-test-slave-bb3x krb5kdc[29908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:49 dist-test-slave-bb3x krb5kdc[29908](info): set up 2 sockets
Sep 23 15:10:49 dist-test-slave-bb3x krb5kdc[29908](info): commencing operation
krb5kdc: starting...
W20260923 15:10:50.587918 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:50 dist-test-slave-bb3x krb5kdc[29908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:50.722600 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:43659
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:43659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:50.805823 29921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:50.806057 29921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:50.806098 29921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:50.807265 29921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:50.807333 29921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:50.807358 29921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:50.807389 29921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:50.807415 29921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:50.808976 29921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:43659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:43659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:50.809255 29921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:50.809492 29921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:50.814183 29926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:50.814276 29927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:50.814410 29929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:50.814417 29921 server_base.cc:1034] running on GCE node
I20260923 15:10:50.814775 29921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:50.815054 29921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:50.816218 29921 hybrid_clock.cc:648] HybridClock initialized: now 1790176250816205 us; error 41 us; skew 500 ppm
Sep 23 15:10:50 dist-test-slave-bb3x krb5kdc[29908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176250, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:50.818776 29921 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:50.819192 29921 webserver.cc:460] Webserver started at http://127.7.250.126:35269/ using document root <none> and password file <none>
I20260923 15:10:50.819442 29921 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:50.819511 29921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:50.819655 29921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:50.820673 29921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "9488bc865b2444c4a7203d6312d6778d"
format_stamp: "Formatted at 2026-09-23 15:10:50 on dist-test-slave-bb3x"
server_key: "8c2a27755050267df2e8c8b893291992"
server_key_iv: "30e51d640112f0f7042f756586623798"
server_key_version: "encryptionkey@0"
I20260923 15:10:50.821053 29921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "9488bc865b2444c4a7203d6312d6778d"
format_stamp: "Formatted at 2026-09-23 15:10:50 on dist-test-slave-bb3x"
server_key: "8c2a27755050267df2e8c8b893291992"
server_key_iv: "30e51d640112f0f7042f756586623798"
server_key_version: "encryptionkey@0"
I20260923 15:10:50.822487 29921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:50.823295 29936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:50.823478 29921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:50.823585 29921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "9488bc865b2444c4a7203d6312d6778d"
format_stamp: "Formatted at 2026-09-23 15:10:50 on dist-test-slave-bb3x"
server_key: "8c2a27755050267df2e8c8b893291992"
server_key_iv: "30e51d640112f0f7042f756586623798"
server_key_version: "encryptionkey@0"
I20260923 15:10:50.823657 29921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:50.845181 29921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:50.847644 29921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:50.847821 29921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:50.851955 29921 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:43659
I20260923 15:10:50.851994 29988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:43659 every 8 connection(s)
I20260923 15:10:50.852386 29921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:50.853283 29989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:50.855540 29989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d: Bootstrap starting.
I20260923 15:10:50.856158 29989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:50.856442 29989 log.cc:824] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:50.857262 29989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d: No bootstrap required, opened a new log
I20260923 15:10:50.858701 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 29921
I20260923 15:10:50.858857 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:50.859036 8169 external_mini_cluster.cc:1428] Setting key a6000d5f7a7a0c57d8c2e292b90333b8
I20260923 15:10:50.859633 29989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9488bc865b2444c4a7203d6312d6778d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 43659 } }
I20260923 15:10:50.859840 29989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:50.859882 29989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9488bc865b2444c4a7203d6312d6778d, State: Initialized, Role: FOLLOWER
I20260923 15:10:50.860008 29989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9488bc865b2444c4a7203d6312d6778d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 43659 } }
I20260923 15:10:50.860083 29989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:50.860116 29989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:50.860221 29989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:50.861359 29989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9488bc865b2444c4a7203d6312d6778d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 43659 } }
I20260923 15:10:50.861464 29989 leader_election.cc:302] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9488bc865b2444c4a7203d6312d6778d; no voters:
I20260923 15:10:50.861604 29989 leader_election.cc:288] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:50.861662 29994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:50.861840 29989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:50.861847 29994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [term 1 LEADER]: Becoming Leader. State: Replica: 9488bc865b2444c4a7203d6312d6778d, State: Running, Role: LEADER
I20260923 15:10:50.862908 29994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9488bc865b2444c4a7203d6312d6778d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 43659 } }
I20260923 15:10:50.863442 29995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9488bc865b2444c4a7203d6312d6778d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9488bc865b2444c4a7203d6312d6778d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 43659 } } }
I20260923 15:10:50.863538 29995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:50.863442 29996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9488bc865b2444c4a7203d6312d6778d. Latest consensus state: current_term: 1 leader_uuid: "9488bc865b2444c4a7203d6312d6778d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9488bc865b2444c4a7203d6312d6778d" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 43659 } } }
I20260923 15:10:50.864358 29996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d [sys.catalog]: This master's current role is: LEADER
W20260923 15:10:50.864212 30008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:10:50.864411 30008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:50.864500 30011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:50.864694 30011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:50.865221 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:50.859601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34573 (local address 127.7.250.126:43659)
0923 15:10:50.859836 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:50.859846 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:50.859880 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:10:50.859992 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:50.859999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:50.860053 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:50.860200 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:50.860488 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:50.861935 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:50.862841 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:50.863504 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:50.863622 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:50.864480 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:50.864485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:50.864485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:50.864712 (+ 227us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:50.864750 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:50.864765 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:50.864850 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:50.864983 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 15:10:50.865016 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":132,"threads_started":1}
I20260923 15:10:50.866639 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:50.866670 30011 catalog_manager.cc:1347] Generated new cluster ID: 97e349ad37734d86ab4ec3dc7459c1b8
I20260923 15:10:50.866716 30011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:50.871727 29993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:50.868094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35251 (local address 127.7.250.126:43659)
0923 15:10:50.868142 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:50.868143 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:50.868337 (+ 194us) server_negotiation.cc:407] Connection header received
0923 15:10:50.868399 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:50.868400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:50.868410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:50.868465 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:50.868590 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:50.869292 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:50.870335 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:50.870643 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:50.870705 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:50.871480 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:50.871481 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:50.871481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:50.871486 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:50.871499 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:50.871503 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:50.871557 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:50.871637 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 15:10:50.871649 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:10:50.875454 30011 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:50.875959 30011 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:50.888932 30011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9488bc865b2444c4a7203d6312d6778d: Generated new TSK 0
I20260923 15:10:50.889145 30011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:50.893767 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 29921
W20260923 15:10:50.898394 29906 connection.cc:535] client connection to 127.7.250.126:43659 recv error: Network error: failed to read from TLS socket (remote: 127.7.250.126:43659): Connection reset by peer (error 104)
2026-09-23T15:10:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:50 dist-test-slave-bb3x krb5kdc[30020](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:50 dist-test-slave-bb3x krb5kdc[30020](info): set up 2 sockets
Sep 23 15:10:50 dist-test-slave-bb3x krb5kdc[30020](info): commencing operation
krb5kdc: starting...
W20260923 15:10:51.955964 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:52 dist-test-slave-bb3x krb5kdc[30020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:52.077649 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39219
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:39657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:52.158135 30033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:52.158401 30033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:52.158442 30033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:52.159592 30033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:52.159664 30033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:52.159689 30033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:52.159715 30033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:52.159746 30033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:52.161212 30033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:39657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39219
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:52.161490 30033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:52.161731 30033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:52.166921 30039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:52.167021 30033 server_base.cc:1034] running on GCE node
W20260923 15:10:52.166990 30041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:52.166917 30038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:52.167560 30033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:52.167933 30033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:52.169118 30033 hybrid_clock.cc:648] HybridClock initialized: now 1790176252169092 us; error 40 us; skew 500 ppm
Sep 23 15:10:52 dist-test-slave-bb3x krb5kdc[30020](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176252, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:52.178417 30033 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:52.178893 30033 webserver.cc:460] Webserver started at http://127.7.250.126:40741/ using document root <none> and password file <none>
I20260923 15:10:52.179152 30033 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:52.179224 30033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:52.179375 30033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:52.180434 30033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "47b5d787793947b78c405aeeba0e28b3"
format_stamp: "Formatted at 2026-09-23 15:10:52 on dist-test-slave-bb3x"
server_key: "24c21d166489a0b24ee32233dc179ec5"
server_key_iv: "a28bdc06f02bfce4d1acf495da73fc85"
server_key_version: "encryptionkey@0"
I20260923 15:10:52.180791 30033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "47b5d787793947b78c405aeeba0e28b3"
format_stamp: "Formatted at 2026-09-23 15:10:52 on dist-test-slave-bb3x"
server_key: "24c21d166489a0b24ee32233dc179ec5"
server_key_iv: "a28bdc06f02bfce4d1acf495da73fc85"
server_key_version: "encryptionkey@0"
I20260923 15:10:52.182204 30033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:52.183163 30048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:52.183534 30033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:10:52.183672 30033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "47b5d787793947b78c405aeeba0e28b3"
format_stamp: "Formatted at 2026-09-23 15:10:52 on dist-test-slave-bb3x"
server_key: "24c21d166489a0b24ee32233dc179ec5"
server_key_iv: "a28bdc06f02bfce4d1acf495da73fc85"
server_key_version: "encryptionkey@0"
I20260923 15:10:52.183758 30033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:52.187286 30033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:52.189656 30033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:52.189817 30033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:52.193861 30033 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:39219
I20260923 15:10:52.193931 30100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:39219 every 8 connection(s)
I20260923 15:10:52.194341 30033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:52.195439 30101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:52.197893 30101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3: Bootstrap starting.
I20260923 15:10:52.198616 30101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:52.198938 30101 log.cc:824] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:52.199775 30101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3: No bootstrap required, opened a new log
I20260923 15:10:52.201635 30101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b5d787793947b78c405aeeba0e28b3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39219 } }
I20260923 15:10:52.201800 30101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:52.201828 30101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47b5d787793947b78c405aeeba0e28b3, State: Initialized, Role: FOLLOWER
I20260923 15:10:52.202040 30101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b5d787793947b78c405aeeba0e28b3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39219 } }
I20260923 15:10:52.202118 30101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:52.202142 30101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:52.202198 30101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:52.202905 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30033
I20260923 15:10:52.203056 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:52.203303 30101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b5d787793947b78c405aeeba0e28b3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39219 } }
I20260923 15:10:52.203329 8169 external_mini_cluster.cc:1428] Setting key 0ee8373c4ea38a9864c90819f63db4ef
I20260923 15:10:52.203397 30101 leader_election.cc:302] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47b5d787793947b78c405aeeba0e28b3; no voters:
I20260923 15:10:52.203595 30101 leader_election.cc:288] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:52.203619 30104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:52.203750 30104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [term 1 LEADER]: Becoming Leader. State: Replica: 47b5d787793947b78c405aeeba0e28b3, State: Running, Role: LEADER
I20260923 15:10:52.203892 30101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:52.203900 30104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b5d787793947b78c405aeeba0e28b3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39219 } }
I20260923 15:10:52.204398 30104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47b5d787793947b78c405aeeba0e28b3. Latest consensus state: current_term: 1 leader_uuid: "47b5d787793947b78c405aeeba0e28b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b5d787793947b78c405aeeba0e28b3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39219 } } }
I20260923 15:10:52.204491 30104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:52.204941 30105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47b5d787793947b78c405aeeba0e28b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47b5d787793947b78c405aeeba0e28b3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39219 } } }
I20260923 15:10:52.205072 30105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:52.205353 30112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:52.205936 30112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:52.206522 30108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:52.204116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43677 (local address 127.7.250.126:39219)
0923 15:10:52.204389 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:52.204401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:52.204437 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:10:52.204566 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:52.204574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:52.204621 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:52.205148 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:52.205824 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:52.205827 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:52.205827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:52.205830 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:52.205858 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:52.205869 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:52.205953 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:52.206281 (+ 328us) server_negotiation.cc:299] Negotiation successful
0923 15:10:52.206334 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":71,"threads_started":1}
I20260923 15:10:52.207758 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:52.207825 30112 catalog_manager.cc:1347] Generated new cluster ID: 6e89022a976f486a95e834b1ab09a8a1
I20260923 15:10:52.207864 30112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:52.210536 30108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:52.209323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35493 (local address 127.7.250.126:39219)
0923 15:10:52.209357 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:52.209359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:52.209496 (+ 137us) server_negotiation.cc:407] Connection header received
0923 15:10:52.209572 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:52.209573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:52.209579 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:52.209640 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:52.210304 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:52.210306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:52.210306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:52.210310 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:52.210326 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:52.210330 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:52.210390 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:52.210456 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 15:10:52.210468 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:10:52.217492 30112 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:52.217733 30112 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:52.221698 30112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47b5d787793947b78c405aeeba0e28b3: Generated new TSK 0
I20260923 15:10:52.221938 30112 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:52.224308 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30033
2026-09-23T15:10:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:52 dist-test-slave-bb3x krb5kdc[30132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:52 dist-test-slave-bb3x krb5kdc[30132](info): set up 2 sockets
Sep 23 15:10:52 dist-test-slave-bb3x krb5kdc[30132](info): commencing operation
krb5kdc: starting...
W20260923 15:10:53.275988 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:53 dist-test-slave-bb3x krb5kdc[30132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:53.402074 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:38745
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:38745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:53.485747 30145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:53.486032 30145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:53.486102 30145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:53.488051 30145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:53.488140 30145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:53.488168 30145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:53.488197 30145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:53.488232 30145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:53.489967 30145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:38745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:38745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:53.490300 30145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:53.490576 30145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:53.495769 30153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:53.495779 30150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:53.495908 30145 server_base.cc:1034] running on GCE node
W20260923 15:10:53.495890 30151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:53.496269 30145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:53.496533 30145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:53.497686 30145 hybrid_clock.cc:648] HybridClock initialized: now 1790176253497661 us; error 38 us; skew 500 ppm
Sep 23 15:10:53 dist-test-slave-bb3x krb5kdc[30132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176253, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:53.500360 30145 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:53.500809 30145 webserver.cc:460] Webserver started at http://127.7.250.126:39381/ using document root <none> and password file <none>
I20260923 15:10:53.501017 30145 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:53.501057 30145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:53.501157 30145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:53.502204 30145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159"
format_stamp: "Formatted at 2026-09-23 15:10:53 on dist-test-slave-bb3x"
server_key: "ae61640116874df1f0be42211275ece3"
server_key_iv: "becd80c6180fd9b2b7d5e9ab1485ff0b"
server_key_version: "encryptionkey@0"
I20260923 15:10:53.502593 30145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159"
format_stamp: "Formatted at 2026-09-23 15:10:53 on dist-test-slave-bb3x"
server_key: "ae61640116874df1f0be42211275ece3"
server_key_iv: "becd80c6180fd9b2b7d5e9ab1485ff0b"
server_key_version: "encryptionkey@0"
I20260923 15:10:53.503912 30145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:53.504813 30160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:53.505028 30145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:53.505117 30145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159"
format_stamp: "Formatted at 2026-09-23 15:10:53 on dist-test-slave-bb3x"
server_key: "ae61640116874df1f0be42211275ece3"
server_key_iv: "becd80c6180fd9b2b7d5e9ab1485ff0b"
server_key_version: "encryptionkey@0"
I20260923 15:10:53.505172 30145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:53.518411 30145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:53.520284 30145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:53.520395 30145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:53.524627 30145 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:38745
I20260923 15:10:53.524736 30212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:38745 every 8 connection(s)
I20260923 15:10:53.525066 30145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:53.526001 30213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:53.527253 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30145
I20260923 15:10:53.527339 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:53.527539 8169 external_mini_cluster.cc:1428] Setting key 844b4e2b3cad67dbda94680b385fc6c9
I20260923 15:10:53.529366 30213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159: Bootstrap starting.
I20260923 15:10:53.530092 30213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:53.530505 30213 log.cc:824] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:53.531333 30213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159: No bootstrap required, opened a new log
I20260923 15:10:53.534052 30213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38745 } }
I20260923 15:10:53.534212 30213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:53.534240 30213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd3e5d9b8ce410bb3ff0fed04ac2159, State: Initialized, Role: FOLLOWER
I20260923 15:10:53.534459 30213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38745 } }
I20260923 15:10:53.534548 30213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:53.534601 30213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:53.534665 30213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:53.535463 30213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38745 } }
I20260923 15:10:53.535609 30213 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd3e5d9b8ce410bb3ff0fed04ac2159; no voters:
Sep 23 15:10:53 dist-test-slave-bb3x krb5kdc[30132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:53.535804 30213 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:53.535879 30218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:53.536072 30218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [term 1 LEADER]: Becoming Leader. State: Replica: 1dd3e5d9b8ce410bb3ff0fed04ac2159, State: Running, Role: LEADER
I20260923 15:10:53.536095 30213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:53.536204 30218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38745 } }
I20260923 15:10:53.536556 30218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd3e5d9b8ce410bb3ff0fed04ac2159. Latest consensus state: current_term: 1 leader_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38745 } } }
I20260923 15:10:53.536725 30219 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd3e5d9b8ce410bb3ff0fed04ac2159" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 38745 } } }
I20260923 15:10:53.536794 30219 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:53.536651 30218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:53.537403 30227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:53.537905 30227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:53.539700 30227 catalog_manager.cc:1347] Generated new cluster ID: 2ee5d40ef8fc43288014ed2fa7c43732
I20260923 15:10:53.539765 30227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:53.540947 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:53.528090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42935 (local address 127.7.250.126:38745)
0923 15:10:53.528287 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:53.528298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:53.528367 (+ 69us) server_negotiation.cc:407] Connection header received
0923 15:10:53.529040 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:53.529051 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:53.529119 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:53.529301 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:53.530086 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:53.531644 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:53.532438 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:53.533170 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:53.533329 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:53.536534 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:53.536539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:53.536539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:53.536553 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:53.539199 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:53.539617 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:53.539619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:53.539620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:53.539724 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:53.539947 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:53.539948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:53.539949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:53.540296 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:53.540452 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:53.540583 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 15:10:53.540726 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":59,"threads_started":1}
I20260923 15:10:53.541993 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:53.551370 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:53.543585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35215 (local address 127.7.250.126:38745)
0923 15:10:53.543624 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:53.543626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:53.543637 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:10:53.543677 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:53.543677 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:53.543688 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:53.543753 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:53.544485 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:53.545302 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:53.546392 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:53.546767 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:53.546842 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:53.548228 (+ 1386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:53.548230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:53.548230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:53.548238 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:53.549984 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:53.550425 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:53.550426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:53.550426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:53.550492 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:53.550707 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:53.550708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:53.550708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:53.550841 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:53.550940 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:53.551072 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 15:10:53.551157 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:10:53.559943 30227 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:53.560554 30227 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:53.566519 30227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd3e5d9b8ce410bb3ff0fed04ac2159: Generated new TSK 0
I20260923 15:10:53.566709 30227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:53.571841 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30145
2026-09-23T15:10:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:53 dist-test-slave-bb3x krb5kdc[30244](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:53 dist-test-slave-bb3x krb5kdc[30244](info): set up 2 sockets
Sep 23 15:10:53 dist-test-slave-bb3x krb5kdc[30244](info): commencing operation
krb5kdc: starting...
W20260923 15:10:54.664187 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:54 dist-test-slave-bb3x krb5kdc[30244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:54.789736 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42853
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:42975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:42853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:54.870668 30257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:54.870929 30257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:54.870970 30257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:54.872189 30257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:54.872260 30257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:54.872284 30257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:54.872313 30257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:54.872340 30257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:54.873842 30257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:42975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:42853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:42853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:54.874137 30257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:54.874389 30257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:54.879432 30262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:54.879423 30265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:54.879423 30263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:54.879660 30257 server_base.cc:1034] running on GCE node
I20260923 15:10:54.879945 30257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:54.880268 30257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:54.881459 30257 hybrid_clock.cc:648] HybridClock initialized: now 1790176254881435 us; error 34 us; skew 500 ppm
Sep 23 15:10:54 dist-test-slave-bb3x krb5kdc[30244](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176254, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:54.884325 30257 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:54.884768 30257 webserver.cc:460] Webserver started at http://127.7.250.126:40669/ using document root <none> and password file <none>
I20260923 15:10:54.885016 30257 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:54.885067 30257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:54.885253 30257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:54.886490 30257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "29c9ec08c510425c835bb9cfc316c31b"
format_stamp: "Formatted at 2026-09-23 15:10:54 on dist-test-slave-bb3x"
server_key: "1c3c0abc3b9ee35d660bf91758dedde3"
server_key_iv: "45b291a3c90f566b46dbd61045aa2774"
server_key_version: "encryptionkey@0"
I20260923 15:10:54.886878 30257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "29c9ec08c510425c835bb9cfc316c31b"
format_stamp: "Formatted at 2026-09-23 15:10:54 on dist-test-slave-bb3x"
server_key: "1c3c0abc3b9ee35d660bf91758dedde3"
server_key_iv: "45b291a3c90f566b46dbd61045aa2774"
server_key_version: "encryptionkey@0"
I20260923 15:10:54.888396 30257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:10:54.889310 30272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:54.889576 30257 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:54.889688 30257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "29c9ec08c510425c835bb9cfc316c31b"
format_stamp: "Formatted at 2026-09-23 15:10:54 on dist-test-slave-bb3x"
server_key: "1c3c0abc3b9ee35d660bf91758dedde3"
server_key_iv: "45b291a3c90f566b46dbd61045aa2774"
server_key_version: "encryptionkey@0"
I20260923 15:10:54.889773 30257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:54.925220 30257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:54.927614 30257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:54.927783 30257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:54.932008 30257 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:42853
I20260923 15:10:54.932072 30324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:42853 every 8 connection(s)
I20260923 15:10:54.932422 30257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:54.933308 30325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:54.935153 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30257
I20260923 15:10:54.935294 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:54.935529 8169 external_mini_cluster.cc:1428] Setting key 3616209611b4c9774c21d33d72f4f7c9
I20260923 15:10:54.935877 30325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b: Bootstrap starting.
I20260923 15:10:54.936684 30325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:54.937017 30325 log.cc:824] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:54.937808 30325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b: No bootstrap required, opened a new log
I20260923 15:10:54.939818 30325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c9ec08c510425c835bb9cfc316c31b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42853 } }
I20260923 15:10:54.939980 30325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:54.940068 30325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29c9ec08c510425c835bb9cfc316c31b, State: Initialized, Role: FOLLOWER
I20260923 15:10:54.940217 30325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c9ec08c510425c835bb9cfc316c31b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42853 } }
I20260923 15:10:54.940305 30325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:54.940351 30325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:54.940397 30325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:54.940951 30325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c9ec08c510425c835bb9cfc316c31b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42853 } }
I20260923 15:10:54.941183 30325 leader_election.cc:302] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29c9ec08c510425c835bb9cfc316c31b; no voters:
I20260923 15:10:54.941377 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:54.936137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47689 (local address 127.7.250.126:42853)
0923 15:10:54.936419 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:54.936429 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:54.936465 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:10:54.937117 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:54.937126 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:54.937215 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:54.937410 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:54.937625 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:54.938671 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:54.939468 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:54.939872 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:54.939991 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:54.940701 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:54.940705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:54.940705 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:54.940879 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:54.940910 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:54.940920 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:54.941000 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:54.941076 (+ 76us) server_negotiation.cc:299] Negotiation successful
0923 15:10:54.941141 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":159,"threads_started":1}
I20260923 15:10:54.941462 30325 leader_election.cc:288] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:54.941519 30330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:54.941702 30330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [term 1 LEADER]: Becoming Leader. State: Replica: 29c9ec08c510425c835bb9cfc316c31b, State: Running, Role: LEADER
I20260923 15:10:54.941712 30325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:54.941936 30330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c9ec08c510425c835bb9cfc316c31b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42853 } }
I20260923 15:10:54.942279 30330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29c9ec08c510425c835bb9cfc316c31b. Latest consensus state: current_term: 1 leader_uuid: "29c9ec08c510425c835bb9cfc316c31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c9ec08c510425c835bb9cfc316c31b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42853 } } }
I20260923 15:10:54.942381 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:54.942494 30330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:54.942430 30332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29c9ec08c510425c835bb9cfc316c31b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29c9ec08c510425c835bb9cfc316c31b" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 42853 } } }
I20260923 15:10:54.942627 30332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:54.942732 30338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:54.943199 30338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:54.944834 30338 catalog_manager.cc:1347] Generated new cluster ID: 46552c22098447a0a44429943ae6984f
I20260923 15:10:54.944880 30338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:54.949812 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:54.943910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54133 (local address 127.7.250.126:42853)
0923 15:10:54.943951 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:54.943953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:54.946541 (+ 2588us) server_negotiation.cc:407] Connection header received
0923 15:10:54.946590 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:54.946591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:54.946603 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:54.946659 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:54.946813 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:54.947561 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:54.948508 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:54.948813 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:54.948886 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:54.949567 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:54.949569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:54.949569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:54.949575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:54.949591 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:54.949595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:54.949654 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:54.949710 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 15:10:54.949732 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:10:54.951818 30338 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:54.952365 30338 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:54.965528 30338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29c9ec08c510425c835bb9cfc316c31b: Generated new TSK 0
I20260923 15:10:54.965731 30338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:54.972280 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30257
2026-09-23T15:10:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:55 dist-test-slave-bb3x krb5kdc[30356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:55 dist-test-slave-bb3x krb5kdc[30356](info): set up 2 sockets
Sep 23 15:10:55 dist-test-slave-bb3x krb5kdc[30356](info): commencing operation
krb5kdc: starting...
W20260923 15:10:56.099992 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:56 dist-test-slave-bb3x krb5kdc[30356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:56.225237 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39953
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39953
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:56.307034 30369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:56.307292 30369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:56.307334 30369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:56.308533 30369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:56.308609 30369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:56.308634 30369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:56.308661 30369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:56.308692 30369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:56.310176 30369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:56.310477 30369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:56.310720 30369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:56.315819 30375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:56.315853 30369 server_base.cc:1034] running on GCE node
W20260923 15:10:56.315819 30374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:56.315858 30377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:56.316319 30369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:56.316618 30369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:56.317780 30369 hybrid_clock.cc:648] HybridClock initialized: now 1790176256317749 us; error 45 us; skew 500 ppm
Sep 23 15:10:56 dist-test-slave-bb3x krb5kdc[30356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176256, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:56.320351 30369 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:56.320811 30369 webserver.cc:460] Webserver started at http://127.7.250.126:34667/ using document root <none> and password file <none>
I20260923 15:10:56.320992 30369 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:56.321033 30369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:56.321122 30369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:56.322170 30369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "eb22cf683fce411eb8944e837ad189e9"
format_stamp: "Formatted at 2026-09-23 15:10:56 on dist-test-slave-bb3x"
server_key: "a1b870f0392a7dca3deb70ae84b4b1af"
server_key_iv: "934de0691b0a8767916ed08f17976206"
server_key_version: "encryptionkey@0"
I20260923 15:10:56.322556 30369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "eb22cf683fce411eb8944e837ad189e9"
format_stamp: "Formatted at 2026-09-23 15:10:56 on dist-test-slave-bb3x"
server_key: "a1b870f0392a7dca3deb70ae84b4b1af"
server_key_iv: "934de0691b0a8767916ed08f17976206"
server_key_version: "encryptionkey@0"
I20260923 15:10:56.323906 30369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:56.324676 30384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:56.324865 30369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:56.324973 30369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "eb22cf683fce411eb8944e837ad189e9"
format_stamp: "Formatted at 2026-09-23 15:10:56 on dist-test-slave-bb3x"
server_key: "a1b870f0392a7dca3deb70ae84b4b1af"
server_key_iv: "934de0691b0a8767916ed08f17976206"
server_key_version: "encryptionkey@0"
I20260923 15:10:56.325052 30369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:56.328464 30369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:56.330701 30369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:56.330865 30369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:56.334942 30369 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:39953
I20260923 15:10:56.334975 30436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:39953 every 8 connection(s)
I20260923 15:10:56.335372 30369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:56.336421 30437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:56.338788 30437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9: Bootstrap starting.
I20260923 15:10:56.339378 30437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:56.339682 30437 log.cc:824] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:56.340464 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30369
I20260923 15:10:56.340593 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:56.340660 30437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9: No bootstrap required, opened a new log
I20260923 15:10:56.340802 8169 external_mini_cluster.cc:1428] Setting key 8b925ada130057e017c15a84ae9e9b85
I20260923 15:10:56.342661 30437 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb22cf683fce411eb8944e837ad189e9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39953 } }
I20260923 15:10:56.342833 30437 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:10:56.342898 30437 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb22cf683fce411eb8944e837ad189e9, State: Initialized, Role: FOLLOWER
I20260923 15:10:56.343072 30437 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb22cf683fce411eb8944e837ad189e9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39953 } }
I20260923 15:10:56.343503 30437 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:10:56.343561 30437 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:10:56.343618 30437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:56.344210 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:56.341428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54917 (local address 127.7.250.126:39953)
0923 15:10:56.341635 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:56.341648 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:56.341674 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:10:56.342663 (+ 989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:56.342671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:56.342703 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:56.342858 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:56.343654 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:56.343658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:56.343659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:56.343663 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:56.343711 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:56.343726 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:56.343787 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:56.343895 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 15:10:56.343951 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260923 15:10:56.344372 30437 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb22cf683fce411eb8944e837ad189e9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39953 } }
I20260923 15:10:56.344484 30437 leader_election.cc:302] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb22cf683fce411eb8944e837ad189e9; no voters:
I20260923 15:10:56.344641 30437 leader_election.cc:288] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:10:56.344708 30442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:56.344843 30442 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [term 1 LEADER]: Becoming Leader. State: Replica: eb22cf683fce411eb8944e837ad189e9, State: Running, Role: LEADER
I20260923 15:10:56.344908 30437 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:56.344997 30442 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb22cf683fce411eb8944e837ad189e9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39953 } }
I20260923 15:10:56.345230 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:56.345425 30444 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb22cf683fce411eb8944e837ad189e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb22cf683fce411eb8944e837ad189e9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39953 } } }
I20260923 15:10:56.345418 30442 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb22cf683fce411eb8944e837ad189e9. Latest consensus state: current_term: 1 leader_uuid: "eb22cf683fce411eb8944e837ad189e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb22cf683fce411eb8944e837ad189e9" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39953 } } }
I20260923 15:10:56.345510 30444 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:56.345520 30442 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:56.346392 30451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:56.347414 30451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:56.348026 30441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:56.346852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46273 (local address 127.7.250.126:39953)
0923 15:10:56.346886 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:56.346888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:56.346990 (+ 102us) server_negotiation.cc:407] Connection header received
0923 15:10:56.347041 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:56.347041 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:56.347047 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:56.347118 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:56.347800 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:56.347801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:56.347802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:10:56.347806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:56.347821 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:10:56.347825 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:56.347888 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:56.347951 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 15:10:56.347966 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260923 15:10:56.349179 30451 catalog_manager.cc:1347] Generated new cluster ID: 4752c1ded72046f4b786672572572d4c
I20260923 15:10:56.349231 30451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:56.358901 30451 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:56.359102 30451 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:56.366357 30451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb22cf683fce411eb8944e837ad189e9: Generated new TSK 0
I20260923 15:10:56.366626 30451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:10:56.369947 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30369
2026-09-23T15:10:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1398 ms)
[----------] 18 tests from AuthTokenIssuingTest (24490 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:56 dist-test-slave-bb3x krb5kdc[30468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:56 dist-test-slave-bb3x krb5kdc[30468](info): set up 2 sockets
Sep 23 15:10:56 dist-test-slave-bb3x krb5kdc[30468](info): commencing operation
krb5kdc: starting...
W20260923 15:10:57.412187 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:10:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:10:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:10:57.535667 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39251
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39251,127.7.250.125:44847,127.7.250.124:38373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:10:57.618798 30481 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:10:57.619012 30481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:57.619029 30481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:57.619050 30481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:57.620329 30481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:57.620369 30481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:57.620384 30481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:57.620397 30481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:57.620445 30481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:57.622066 30481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39251,127.7.250.125:44847,127.7.250.124:38373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:39251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:57.622368 30481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:57.622601 30481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:57.627525 30489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:57.627565 30486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:57.627501 30487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:57.627863 30481 server_base.cc:1034] running on GCE node
I20260923 15:10:57.628249 30481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:57.628562 30481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:57.629727 30481 hybrid_clock.cc:648] HybridClock initialized: now 1790176257629695 us; error 48 us; skew 500 ppm
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:57.637678 30481 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:10:57.638206 30481 webserver.cc:460] Webserver started at http://127.7.250.126:42211/ using document root <none> and password file <none>
I20260923 15:10:57.638500 30481 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:57.638548 30481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:57.638687 30481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:57.639803 30481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "57996bcbda0c49d781e886e8ee010cf3"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "0266dfe31eceb4d3e16e7a04f52f8fb3"
server_key_iv: "a1b60e3df36ad472cebb892ebb16f5b5"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.640156 30481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "57996bcbda0c49d781e886e8ee010cf3"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "0266dfe31eceb4d3e16e7a04f52f8fb3"
server_key_iv: "a1b60e3df36ad472cebb892ebb16f5b5"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.641623 30481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:57.642822 30496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:57.643225 30481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:57.643335 30481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "57996bcbda0c49d781e886e8ee010cf3"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "0266dfe31eceb4d3e16e7a04f52f8fb3"
server_key_iv: "a1b60e3df36ad472cebb892ebb16f5b5"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.643416 30481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:57.652841 30481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:57.654899 30481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:57.655062 30481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:57.659490 30481 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:39251
I20260923 15:10:57.659526 30548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:39251 every 8 connection(s)
I20260923 15:10:57.659917 30481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:10:57.660893 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30481
I20260923 15:10:57.660933 30549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:57.660993 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:10:57.661332 8169 external_mini_cluster.cc:1428] Setting key 284cf5c934e49ef9cb44502edf05a599
I20260923 15:10:57.662714 30549 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.250.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.125@KRBTEST.COM" created.
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:57.675802 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.663447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39251 (local address 127.0.0.1:42700)
0923 15:10:57.663629 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.663671 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:57.664187 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:57.664481 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:57.664484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:57.664769 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:57.664922 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.664926 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.665955 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.665956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.666696 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.666699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.667239 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.667241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.667304 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.667862 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:57.667870 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:57.671933 (+ 4063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:57.674168 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.674175 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.674177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.674434 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.674694 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.674695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.674695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.674786 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.675179 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:57.675185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:57.675251 (+ 66us) client_negotiation.cc:769] Sending connection context
0923 15:10:57.675357 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 15:10:57.675616 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":98,"thread_start_us":83,"threads_started":1}
I20260923 15:10:57.675804 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.663479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42700 (local address 127.7.250.126:39251)
0923 15:10:57.663660 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:57.663671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:57.663780 (+ 109us) server_negotiation.cc:407] Connection header received
0923 15:10:57.664346 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:57.664348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:57.664369 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:57.664478 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:57.665037 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.665842 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.666794 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.667122 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.667252 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.672149 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:57.672158 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:57.672158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:57.672167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:57.674065 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.674537 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.674540 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.674540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.674624 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.674858 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.674858 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.674859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.675106 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:57.675200 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:57.675401 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 15:10:57.675541 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":77,"threads_started":1}
I20260923 15:10:57.677277 30549 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:57.677706 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.677571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32886)
0923 15:10:57.677630 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.677683 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260923 15:10:57.677845 30499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
I20260923 15:10:57.678280 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.678164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32892)
0923 15:10:57.678211 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.678235 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:10:57.678386 30549 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:44847: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab.
Entry for principal kudu/127.7.250.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab.
I20260923 15:10:57.690703 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:44847
--webserver_interface=127.7.250.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab
--principal=kudu/127.7.250.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39251,127.7.250.125:44847,127.7.250.124:38373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:10:57.727639 30549 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } attempt: 1
I20260923 15:10:57.728232 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.728070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32900)
0923 15:10:57.728152 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.728198 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260923 15:10:57.728705 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.728536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32902)
0923 15:10:57.728640 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.728681 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260923 15:10:57.728830 30549 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:44847: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
W20260923 15:10:57.773171 30556 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:10:57.773439 30556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:57.773473 30556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:57.773515 30556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:57.774709 30556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:57.774782 30556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:57.774809 30556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:57.774842 30556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:57.774868 30556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:57.776451 30556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39251,127.7.250.125:44847,127.7.250.124:38373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab
--principal=kudu/127.7.250.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:44847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.250.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:57.776742 30556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:57.776979 30556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:57.782055 30564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:57.782075 30561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:57.782037 30562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:57.782343 30556 server_base.cc:1034] running on GCE node
I20260923 15:10:57.782641 30556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:57.782968 30556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:57.784114 30556 hybrid_clock.cc:648] HybridClock initialized: now 1790176257784098 us; error 26 us; skew 500 ppm
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:57.786919 30556 init.cc:377] Logged in from keytab as kudu/127.7.250.125@KRBTEST.COM (short username kudu)
I20260923 15:10:57.787422 30556 webserver.cc:460] Webserver started at http://127.7.250.125:33325/ using document root <none> and password file <none>
I20260923 15:10:57.787678 30556 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:57.787724 30556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:57.787865 30556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:57.788928 30556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data/instance:
uuid: "5adcbd47b3ce4963918c7b81dc45b4e6"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "3c179a9718f1ccea35e6489fda686af0"
server_key_iv: "073d3133a372451528d670e0ee911dbc"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.789291 30556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal/instance:
uuid: "5adcbd47b3ce4963918c7b81dc45b4e6"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "3c179a9718f1ccea35e6489fda686af0"
server_key_iv: "073d3133a372451528d670e0ee911dbc"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.790704 30556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:57.791424 30571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:57.791610 30556 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:10:57.791673 30556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal
uuid: "5adcbd47b3ce4963918c7b81dc45b4e6"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "3c179a9718f1ccea35e6489fda686af0"
server_key_iv: "073d3133a372451528d670e0ee911dbc"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.791764 30556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:57.797015 30549 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } attempt: 2
I20260923 15:10:57.797487 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.797327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32912)
0923 15:10:57.797407 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.797448 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260923 15:10:57.797911 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.797791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32926)
0923 15:10:57.797854 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.797885 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 15:10:57.798138 30549 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:44847: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44847: connect: Connection refused (error 111)
I20260923 15:10:57.823467 30556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:57.825821 30556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:57.826000 30556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:57.829833 30556 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.125:44847
I20260923 15:10:57.829876 30623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.125:44847 every 8 connection(s)
I20260923 15:10:57.830278 30556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
I20260923 15:10:57.831322 30624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:57.832911 30624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:57.836129 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30556
I20260923 15:10:57.836246 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-1/wal/instance
I20260923 15:10:57.836436 8169 external_mini_cluster.cc:1428] Setting key 163db0bd32dbe6c01fcc62b5f04240da
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.124@KRBTEST.COM; defaulting to no policy
I20260923 15:10:57.843644 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.833670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.7.250.126:39251)
0923 15:10:57.833798 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:57.833801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:57.834059 (+ 258us) server_negotiation.cc:407] Connection header received
0923 15:10:57.834862 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:57.834863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:57.834881 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:57.834967 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:57.835685 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.836322 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.837546 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.837806 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.837911 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.840649 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:57.840651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:57.840651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:57.840659 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:57.842157 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.842635 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.842636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.842637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.842702 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.842961 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.842962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.842963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.843093 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:57.843166 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:57.843456 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 15:10:57.843543 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 15:10:57.843829 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.833571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39251 (local address 127.0.0.1:42712)
0923 15:10:57.833854 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.833897 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:57.834734 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:57.835050 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:57.835054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:57.835335 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:57.835558 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.835563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.836457 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.836458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.837419 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.837421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.837951 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.837952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.838094 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.838740 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:57.838749 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:57.840544 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:57.842289 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.842296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.842298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.842536 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.842771 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.842772 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.842772 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.842885 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.843168 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:57.843176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:57.843322 (+ 146us) client_negotiation.cc:769] Sending connection context
0923 15:10:57.843423 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 15:10:57.843584 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":82,"threads_started":1}
I20260923 15:10:57.844424 30624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.124@KRBTEST.COM" created.
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:10:57.852360 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.844695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32932)
0923 15:10:57.844747 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.844759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:57.844799 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:57.845019 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:57.845020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:57.845129 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:57.845182 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.845183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.846048 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.846049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.846911 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.846914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.847643 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.847644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.847703 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.848155 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:57.848160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:57.849269 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:57.851222 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.851223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.851223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.851407 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.851697 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.851698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.851698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.851758 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.852117 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:57.852118 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:57.852155 (+ 37us) client_negotiation.cc:769] Sending connection context
0923 15:10:57.852211 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:10:57.852295 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 15:10:57.852519 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.844750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32932 (local address 127.7.250.125:44847)
0923 15:10:57.844900 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:57.844904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:57.844919 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:10:57.844938 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:57.844939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:57.844950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:57.845013 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:57.845238 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.845956 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.846990 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.847564 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.847646 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.849351 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:57.849359 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:57.849360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:57.849367 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:57.851098 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.851513 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.851517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.851517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.851584 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.851830 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.851831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.851831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.852035 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:57.852129 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:57.852246 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 15:10:57.852390 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260923 15:10:57.853209 30624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:57.853541 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.853434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54714)
0923 15:10:57.853481 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.853523 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:10:57.853724 30573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
I20260923 15:10:57.854166 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.854062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54730)
0923 15:10:57.854116 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.854149 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 15:10:57.854316 30624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:38373: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab.
Entry for principal kudu/127.7.250.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab.
I20260923 15:10:57.866827 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:38373
--webserver_interface=127.7.250.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab
--principal=kudu/127.7.250.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:38865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:39251,127.7.250.125:44847,127.7.250.124:38373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:10:57.889312 30549 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } attempt: 3
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:10:57.897161 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.889792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32942 (local address 127.7.250.125:44847)
0923 15:10:57.889864 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:57.889866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:57.889878 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:10:57.889931 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:57.889932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:57.889945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:57.890017 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:57.890396 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.890937 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.891716 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.891951 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.892041 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.894374 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:57.894376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:57.894376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:57.894383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:57.895965 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.896372 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.896373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.896373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.896430 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:57.896672 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:57.896673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:57.896673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:57.896807 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:57.896864 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:57.897032 (+ 168us) server_negotiation.cc:299] Negotiation successful
0923 15:10:57.897095 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:10:57.897207 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.889703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32942)
0923 15:10:57.889774 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.889790 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:57.889866 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:57.890022 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:57.890023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:57.890196 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:57.890314 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.890316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.891027 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.891028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.891624 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:57.891626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:57.892043 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:57.892045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:57.892133 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:57.892648 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:57.892654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:57.894234 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:57.896059 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.896060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.896060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.896281 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.896500 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:57.896501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:57.896502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:57.896568 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:57.896873 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:57.896874 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:57.896923 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 15:10:57.897003 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 15:10:57.897107 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 15:10:57.897709 30549 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:57.898064 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.897978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54740)
0923 15:10:57.898013 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.898043 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260923 15:10:57.898406 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.898334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54752)
0923 15:10:57.898360 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.898389 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260923 15:10:57.898494 30549 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:38373: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
I20260923 15:10:57.903573 30624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } attempt: 1
I20260923 15:10:57.903972 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.903856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54764)
0923 15:10:57.903914 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.903949 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:10:57.904310 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.904214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54774)
0923 15:10:57.904262 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.904286 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:10:57.904392 30624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:38373: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
I20260923 15:10:57.929652 30549 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } attempt: 1
I20260923 15:10:57.930137 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.929994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54786)
0923 15:10:57.930058 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.930102 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:10:57.930502 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.930402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54792)
0923 15:10:57.930448 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.930473 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:10:57.930589 30549 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:38373: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
W20260923 15:10:57.952718 30631 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:10:57.952958 30631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:10:57.952986 30631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:10:57.953013 30631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:10:57.954154 30631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:10:57.954221 30631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:10:57.954245 30631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:10:57.954265 30631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:10:57.954330 30631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:10:57.955870 30631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:38865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:39251,127.7.250.125:44847,127.7.250.124:38373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab
--principal=kudu/127.7.250.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:38373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.250.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:10:57.956146 30631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:10:57.956377 30631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:10:57.961460 30636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:10:57.961587 30637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:57.961728 30631 server_base.cc:1034] running on GCE node
W20260923 15:10:57.961460 30639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:10:57.962069 30631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:10:57.962419 30631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:10:57.963572 30631 hybrid_clock.cc:648] HybridClock initialized: now 1790176257963522 us; error 65 us; skew 500 ppm
Sep 23 15:10:57 dist-test-slave-bb3x krb5kdc[30468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:10:57.966342 30631 init.cc:377] Logged in from keytab as kudu/127.7.250.124@KRBTEST.COM (short username kudu)
I20260923 15:10:57.966794 30631 webserver.cc:460] Webserver started at http://127.7.250.124:44241/ using document root <none> and password file <none>
I20260923 15:10:57.967048 30631 fs_manager.cc:359] Metadata directory not provided
I20260923 15:10:57.967094 30631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:10:57.967211 30631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:10:57.968274 30631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data/instance:
uuid: "5fcd2ad0b95c4279a978eb451a95d77b"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "e8bdad8695feaedfa7f4852d495bd6c7"
server_key_iv: "fb80abaf4a805cf59945e2d35780da97"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.968644 30631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal/instance:
uuid: "5fcd2ad0b95c4279a978eb451a95d77b"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "e8bdad8695feaedfa7f4852d495bd6c7"
server_key_iv: "fb80abaf4a805cf59945e2d35780da97"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.970055 30631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:10:57.970971 30646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:10:57.971302 30631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:10:57.971410 30631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal
uuid: "5fcd2ad0b95c4279a978eb451a95d77b"
format_stamp: "Formatted at 2026-09-23 15:10:57 on dist-test-slave-bb3x"
server_key: "e8bdad8695feaedfa7f4852d495bd6c7"
server_key_iv: "fb80abaf4a805cf59945e2d35780da97"
server_key_version: "encryptionkey@0"
I20260923 15:10:57.971490 30631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:10:57.972544 30624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } attempt: 2
I20260923 15:10:57.973011 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.972871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54802)
0923 15:10:57.972936 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.972971 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 15:10:57.973605 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:57.973493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54804)
0923 15:10:57.973546 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:57.973577 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 15:10:57.973713 30624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:38373: Network error: Client connection negotiation failed: client connection to 127.7.250.124:38373: connect: Connection refused (error 111)
I20260923 15:10:57.992595 30631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:10:57.995538 30631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:10:57.995730 30631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:10:58.000003 30631 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.124:38373
I20260923 15:10:58.000046 30698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.124:38373 every 8 connection(s)
I20260923 15:10:58.000427 30631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
I20260923 15:10:58.001540 30699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:10:58.002466 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30631
I20260923 15:10:58.002591 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175983307587-8169-0/minicluster-data/master-2/wal/instance
I20260923 15:10:58.002820 8169 external_mini_cluster.cc:1428] Setting key c29787acbfd484f58ddeaf076371fced
I20260923 15:10:58.003876 30699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:58.005759 30549 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } attempt: 2
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:10:58.014657 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.005996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54826)
0923 15:10:58.006042 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:58.006051 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:58.006087 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:58.006327 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:58.006328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:58.006470 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:58.006504 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.006506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.007456 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.007457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.008267 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.008270 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.008707 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.008708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.008778 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.009165 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:58.009171 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:58.011226 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:58.013220 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.013221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.013221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.013404 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.013680 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.013681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.013682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.013748 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.014210 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:58.014211 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:58.014283 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 15:10:58.014429 (+ 146us) client_negotiation.cc:240] Negotiation successful
0923 15:10:58.014540 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260923 15:10:58.014905 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.006042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.7.250.124:38373)
0923 15:10:58.006187 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.006189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.006196 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:10:58.006214 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.006214 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.006222 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.006319 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.006592 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.007359 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.008330 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.008604 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.008925 (+ 321us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.011311 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.011320 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.011320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.011327 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.013134 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.013497 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.013501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.013501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.013601 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.013813 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.013814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.013814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.014115 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.014222 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.014508 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.014660 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260923 15:10:58.015231 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.003466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54818 (local address 127.7.250.124:38373)
0923 15:10:58.003681 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.003689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.003719 (+ 30us) server_negotiation.cc:407] Connection header received
0923 15:10:58.004255 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.004263 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.004318 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.004452 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.005366 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.006673 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.007690 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.008146 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.008218 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.011517 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.011518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.011519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.011525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.013642 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.014055 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.014056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.014056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.014105 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.014331 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.014332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.014332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.014507 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.015004 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.015046 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.015132 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":116,"threads_started":1}
I20260923 15:10:58.015779 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.005002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42716 (local address 127.7.250.126:39251)
0923 15:10:58.005073 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.005076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.005293 (+ 217us) server_negotiation.cc:407] Connection header received
0923 15:10:58.005366 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.005367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.005379 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.005442 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.005994 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.006725 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.007584 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.007845 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.007925 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.010459 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.010461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.010461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.010483 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.012668 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.014105 (+ 1437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.014106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.014107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.014162 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.014477 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.014478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.014479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.014608 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.014649 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.015572 (+ 923us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.015659 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:10:58.015797 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.004899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:39251 (local address 127.0.0.1:42716)
0923 15:10:58.005171 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:58.005218 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:58.005290 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:58.005466 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:58.005468 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:58.005702 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:58.005855 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.005861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.006823 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.006824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.007501 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.007503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.007955 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.007956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.008065 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.008644 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:58.008654 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:58.010370 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:58.013764 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.013771 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.013773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.014005 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.014319 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.014319 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.014320 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.014398 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.015118 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:58.015125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:58.015202 (+ 77us) client_negotiation.cc:769] Sending connection context
0923 15:10:58.015537 (+ 335us) client_negotiation.cc:240] Negotiation successful
0923 15:10:58.015656 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":123,"threads_started":1}
I20260923 15:10:58.016634 30699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:58.016736 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3: Bootstrap starting.
I20260923 15:10:58.017385 30549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:58.017690 30549 log.cc:824] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:58.018560 30549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3: No bootstrap required, opened a new log
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:10:58.020833 30549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.020990 30549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57996bcbda0c49d781e886e8ee010cf3, State: Initialized, Role: FOLLOWER
I20260923 15:10:58.021072 30549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.021589 30549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:58.021396 30707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.021824 30707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: This master's current role is: FOLLOWER
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
W20260923 15:10:58.024273 30718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:10:58.024338 30718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:58.025272 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.015851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.7.250.126:39251)
0923 15:10:58.015887 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.015889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.016024 (+ 135us) server_negotiation.cc:407] Connection header received
0923 15:10:58.016095 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.016095 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.016111 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.016154 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.016769 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.017344 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.018116 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.018419 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.018485 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.021186 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.021187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.021188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.021195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.023588 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.023993 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.023994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.023994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.024055 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.024274 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.024275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.024276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.024445 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.024608 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.025069 (+ 461us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.025167 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22,"thread_start_us":97,"threads_started":1}
I20260923 15:10:58.026412 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.016901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44847 (local address 127.0.0.1:32952)
0923 15:10:58.016940 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:58.016949 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:58.016980 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:58.018203 (+ 1223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:58.018204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:58.018387 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:58.018432 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.018434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.019284 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.019285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.019797 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.019798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.020315 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.020316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.020401 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.021216 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:58.021222 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:58.022623 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:58.025242 (+ 2619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.025243 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.025243 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.025469 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.025693 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.025694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.025694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.025753 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.026035 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:58.026036 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:58.026080 (+ 44us) client_negotiation.cc:769] Sending connection context
0923 15:10:58.026174 (+ 94us) client_negotiation.cc:240] Negotiation successful
0923 15:10:58.026313 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260923 15:10:58.026429 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.017986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32952 (local address 127.7.250.125:44847)
0923 15:10:58.018044 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.018046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.018055 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:58.018081 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.018082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.018093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.018151 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.018521 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.019184 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.019917 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.020230 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.020300 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.022752 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.022753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.022754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.022762 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.025086 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.025547 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.025548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.025548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.025619 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.025834 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.025835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.025835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.025959 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.026032 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.026220 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.026343 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 15:10:58.026796 30699 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:10:58.024500 30481 master_runner.cc:186] Error getting master registration for 127.7.250.125:44847: OK, instance_id { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" instance_seqno: 1790176257828789 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:10:58.027407 30481 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:10:58.034701 30703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.027057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54838)
0923 15:10:58.027101 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:58.027111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:58.027150 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:58.027258 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:58.027259 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:58.027427 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:58.027466 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.027468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.028311 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.028312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.029118 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.029121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.029651 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.029652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.029711 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.030155 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:58.030160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:58.031645 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:58.033503 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.033504 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.033504 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.033705 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.033881 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.033882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.033882 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.033930 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.034323 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:58.034324 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:58.034367 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 15:10:58.034460 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 15:10:58.034570 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 15:10:58.034705 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.027086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.7.250.124:38373)
0923 15:10:58.027126 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.027128 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.027137 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:58.027205 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.027206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.027215 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.027258 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.027528 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.028222 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.029211 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.029559 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.029637 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.031734 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.031735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.031736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.031742 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.033404 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.033777 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.033778 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.033778 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.033823 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.033986 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.033988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.033988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.034216 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.034332 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.034480 (+ 148us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.034620 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":26}
I20260923 15:10:58.036312 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.028005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.7.250.125:44847)
0923 15:10:58.028047 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.028049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.028436 (+ 387us) server_negotiation.cc:407] Connection header received
0923 15:10:58.028456 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.028456 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.028466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.028506 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.029134 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.029596 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.030549 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.030820 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.030899 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.033218 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.033220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.033220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.033228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.035100 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.035494 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.035495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.035495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.035555 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.035775 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.035776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.035776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.035905 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.036079 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.036139 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.036226 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:10:58.036523 30699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: Bootstrap starting.
I20260923 15:10:58.037344 30699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:58.037658 30699 log.cc:824] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:58.037798 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:10:58.038488 30699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: No bootstrap required, opened a new log
I20260923 15:10:58.038828 30523 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "57996bcbda0c49d781e886e8ee010cf3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42718
I20260923 15:10:58.038987 30523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 15:10:58.039047 30523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:58.039968 30523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.040032 30699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.040220 30523 leader_election.cc:288] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [CANDIDATE]: Term 1 election: Requested vote from peers 5adcbd47b3ce4963918c7b81dc45b4e6 (127.7.250.125:44847), 5fcd2ad0b95c4279a978eb451a95d77b (127.7.250.124:38373)
I20260923 15:10:58.040236 30699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fcd2ad0b95c4279a978eb451a95d77b, State: Initialized, Role: FOLLOWER
I20260923 15:10:58.040330 30699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.040644 30721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.040714 30699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:58.040870 30673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57996bcbda0c49d781e886e8ee010cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5fcd2ad0b95c4279a978eb451a95d77b"
I20260923 15:10:58.041013 30673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:58.042228 30721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:10:58.042246 30673 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57996bcbda0c49d781e886e8ee010cf3 in term 1.
I20260923 15:10:58.042511 30498 leader_election.cc:302] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57996bcbda0c49d781e886e8ee010cf3, 5fcd2ad0b95c4279a978eb451a95d77b; no voters:
I20260923 15:10:58.042665 30707 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:10:58.042798 30707 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [term 1 LEADER]: Becoming Leader. State: Replica: 57996bcbda0c49d781e886e8ee010cf3, State: Running, Role: LEADER
I20260923 15:10:58.042887 30707 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.043290 30707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57996bcbda0c49d781e886e8ee010cf3. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.043370 30707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: This master's current role is: LEADER
W20260923 15:10:58.043006 30732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:10:58.043622 30732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:58.045576 30736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:10:58.045915 30736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:10:58.045285 30631 master_runner.cc:186] Error getting master registration for 127.7.250.125:44847: OK, instance_id { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" instance_seqno: 1790176257828789 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:10:58.047629 30631 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:10:58.052026 30673 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [term 1 FOLLOWER]: Refusing update from remote peer 57996bcbda0c49d781e886e8ee010cf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:58.052549 30733 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:58.053831 30721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57996bcbda0c49d781e886e8ee010cf3. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.053953 30721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:10:58.064693 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.050648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.7.250.126:39251)
0923 15:10:58.050690 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.050693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.050959 (+ 266us) server_negotiation.cc:407] Connection header received
0923 15:10:58.051194 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.051195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.051209 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.051246 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.053010 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.053745 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.055547 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.055842 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.055887 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.059238 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.059239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.059240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.059248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.061058 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.062161 (+ 1103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.062162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.062163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.062236 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.063689 (+ 1453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.063690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.063690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.063840 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.063911 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.064506 (+ 595us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.064592 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:10:58.064785 30624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } attempt: 3
Sep 23 15:10:58 dist-test-slave-bb3x krb5kdc[30468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:10:58.073047 30626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.065059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:38373 (local address 127.0.0.1:54852)
0923 15:10:58.065111 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:10:58.065122 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:10:58.065158 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:10:58.065293 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:10:58.065293 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:10:58.065468 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:10:58.065539 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.065542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.066576 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.066578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.067387 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:10:58.067389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.067929 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.067930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:10:58.067986 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.068430 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:10:58.068435 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:10:58.070010 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:10:58.071875 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.071876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.071877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.072103 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.072341 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:10:58.072342 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:10:58.072342 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:10:58.072410 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:10:58.072726 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:10:58.072726 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:10:58.072783 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 15:10:58.072845 (+ 62us) client_negotiation.cc:240] Negotiation successful
0923 15:10:58.072943 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:10:58.073428 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.065132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54852 (local address 127.7.250.124:38373)
0923 15:10:58.065173 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.065175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.065183 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:10:58.065207 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.065208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.065220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.065282 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.065643 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.066470 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.067480 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.067805 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.067865 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.070106 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.070108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.070108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.070116 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.071768 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.072194 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.072195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.072195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.072265 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.072482 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.072483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.072484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.072627 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.072686 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.073250 (+ 564us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.073332 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:10:58.075322 30624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: Bootstrap starting.
I20260923 15:10:58.076082 30624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: Neither blocks nor log segments found. Creating new log.
I20260923 15:10:58.076406 30624 log.cc:824] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:10:58.077561 30707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.077700 30707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:58.078004 30741 mvcc.cc:205] Tried to move back new op lower bound from 7332561952978739200 to 7332561952944308224. Current Snapshot: MvccSnapshot[applied={T|T < 7332561952978739200}]
I20260923 15:10:58.078275 30721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.078367 30721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:10:58.081614 30624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: No bootstrap required, opened a new log
I20260923 15:10:58.083829 30702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.050873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54840 (local address 127.7.250.124:38373)
0923 15:10:58.050922 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.050924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.057953 (+ 7029us) server_negotiation.cc:407] Connection header received
0923 15:10:58.058278 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.058280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.058292 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.058323 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.062479 (+ 4156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.063244 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.069052 (+ 5808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.069395 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.069436 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.079426 (+ 9990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.079428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.079429 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.079436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.081191 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.082367 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.082368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.082369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.082426 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.082845 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.082846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.082846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.082991 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.083075 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.083652 (+ 577us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.083733 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:10:58.083776 30624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.083925 30624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5adcbd47b3ce4963918c7b81dc45b4e6, State: Initialized, Role: FOLLOWER
I20260923 15:10:58.084003 30624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } }
I20260923 15:10:58.084353 30624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:10:58.084391 30598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "57996bcbda0c49d781e886e8ee010cf3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6"
I20260923 15:10:58.084513 30598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:10:58.085532 30598 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57996bcbda0c49d781e886e8ee010cf3 in term 1.
I20260923 15:10:58.086463 30736 catalog_manager.cc:1347] Generated new cluster ID: ae021018df064682944fb7162f6c5f30
I20260923 15:10:58.086520 30736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:10:58.085800 30597 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [term 1 FOLLOWER]: Refusing update from remote peer 57996bcbda0c49d781e886e8ee010cf3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:10:58.088265 30749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:10:58.089663 30766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.089740 30766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:10:58.089774 30766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57996bcbda0c49d781e886e8ee010cf3. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.089807 30766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:10:58.089828 30766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.089861 30766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:10:58.090315 30628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:58.051025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.7.250.125:44847)
0923 15:10:58.051090 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.051092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.066649 (+ 15557us) server_negotiation.cc:407] Connection header received
0923 15:10:58.066854 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.066855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.066874 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.066940 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.070666 (+ 3726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.071672 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.076896 (+ 5224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:58.077221 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:58.077266 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:58.087082 (+ 9816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:10:58.087083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:10:58.087084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:10:58.087092 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:10:58.088886 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.089335 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.089337 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.089337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.089391 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:10:58.089574 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:10:58.089575 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:10:58.089575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:10:58.089703 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:10:58.089785 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:58.090126 (+ 341us) server_negotiation.cc:299] Negotiation successful
0923 15:10:58.090207 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:10:58.090883 30764 mvcc.cc:205] Tried to move back new op lower bound from 7332561952978739200 to 7332561952944308224. Current Snapshot: MvccSnapshot[applied={T|T < 7332561952978739200}]
I20260923 15:10:58.098896 30762 catalog_manager.cc:1259] Loaded cluster ID: ae021018df064682944fb7162f6c5f30
I20260923 15:10:58.098961 30762 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: loading cluster ID for follower catalog manager: success
W20260923 15:10:58.099038 30762 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260923 15:10:58.099074 30762 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:10:58.103068 30749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "57996bcbda0c49d781e886e8ee010cf3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57996bcbda0c49d781e886e8ee010cf3" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 39251 } } peers { permanent_uuid: "5adcbd47b3ce4963918c7b81dc45b4e6" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44847 } } peers { permanent_uuid: "5fcd2ad0b95c4279a978eb451a95d77b" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 38373 } } }
I20260923 15:10:58.103188 30749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:10:58.105762 30736 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:10:58.106571 30736 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:10:58.120846 30736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57996bcbda0c49d781e886e8ee010cf3: Generated new TSK 0
I20260923 15:10:58.121084 30736 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:10:58.148491 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.146667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32990 (local address 127.7.250.125:44847)
0923 15:10:58.146713 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.146716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.147725 (+ 1009us) server_negotiation.cc:407] Connection header received
0923 15:10:58.147842 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.147844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.147856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.147889 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.148416 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32990: BlockingRecv error: recv got EOF from 127.0.0.1:32990 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:10:58.149504 30553 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.147093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42744 (local address 127.7.250.126:39251)
0923 15:10:58.147135 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.147137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.148922 (+ 1785us) server_negotiation.cc:407] Connection header received
0923 15:10:58.149027 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.149028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.149039 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.149069 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.149420 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42744: BlockingRecv error: recv got EOF from 127.0.0.1:42744 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:10:58.150471 30702 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.147421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54864 (local address 127.7.250.124:38373)
0923 15:10:58.147463 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.147465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.149833 (+ 2368us) server_negotiation.cc:407] Connection header received
0923 15:10:58.149937 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.149938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.149949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.149980 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.150415 (+ 435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54864: BlockingRecv error: recv got EOF from 127.0.0.1:54864 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:10:58.153090 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.152288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32992 (local address 127.7.250.125:44847)
0923 15:10:58.152324 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.152326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.152537 (+ 211us) server_negotiation.cc:407] Connection header received
0923 15:10:58.152647 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.152648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.152669 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.152699 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.153041 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32992: BlockingRecv error: recv got EOF from 127.0.0.1:32992 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:10:58.156206 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.155381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.7.250.125:44847)
0923 15:10:58.155415 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.155417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.155670 (+ 253us) server_negotiation.cc:407] Connection header received
0923 15:10:58.155779 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.155780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.155791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.155823 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.156148 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32996: BlockingRecv error: recv got EOF from 127.0.0.1:32996 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:10:58.160218 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.159402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33006 (local address 127.7.250.125:44847)
0923 15:10:58.159436 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.159438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.159686 (+ 248us) server_negotiation.cc:407] Connection header received
0923 15:10:58.159785 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.159786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.159797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.159825 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.160153 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33006: BlockingRecv error: recv got EOF from 127.0.0.1:33006 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 15:10:58.166504 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.165537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.7.250.125:44847)
0923 15:10:58.165579 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.165581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.165860 (+ 279us) server_negotiation.cc:407] Connection header received
0923 15:10:58.165966 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.165967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.165979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.166010 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.166448 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33010: BlockingRecv error: recv got EOF from 127.0.0.1:33010 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:10:58.176923 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.175967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.7.250.125:44847)
0923 15:10:58.176010 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.176012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.176318 (+ 306us) server_negotiation.cc:407] Connection header received
0923 15:10:58.176429 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.176430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.176443 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.176476 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.176865 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33016: BlockingRecv error: recv got EOF from 127.0.0.1:33016 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 15:10:58.195649 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.194498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.7.250.125:44847)
0923 15:10:58.194544 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.194546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.194852 (+ 306us) server_negotiation.cc:407] Connection header received
0923 15:10:58.194962 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.194963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.194977 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.195008 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.195587 (+ 579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33024: BlockingRecv error: recv got EOF from 127.0.0.1:33024 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:10:58.229923 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.229138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33026 (local address 127.7.250.125:44847)
0923 15:10:58.229194 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.229197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.229325 (+ 128us) server_negotiation.cc:407] Connection header received
0923 15:10:58.229400 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.229402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.229414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.229474 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.229846 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33026: BlockingRecv error: recv got EOF from 127.0.0.1:33026 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 15:10:58.296054 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.295287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33034 (local address 127.7.250.125:44847)
0923 15:10:58.295361 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.295363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.295407 (+ 44us) server_negotiation.cc:407] Connection header received
0923 15:10:58.295491 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.295492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.295503 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.295567 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.295979 (+ 412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33034: BlockingRecv error: recv got EOF from 127.0.0.1:33034 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260923 15:10:58.426056 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.425287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.7.250.125:44847)
0923 15:10:58.425378 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.425382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.425459 (+ 77us) server_negotiation.cc:407] Connection header received
0923 15:10:58.425576 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.425577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.425595 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.425658 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.425994 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33048: BlockingRecv error: recv got EOF from 127.0.0.1:33048 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260923 15:10:58.684029 30628 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:10:58.683394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.7.250.125:44847)
0923 15:10:58.683471 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:58.683473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:58.683482 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:58.683509 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:58.683510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:58.683520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:58.683570 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:10:58.683959 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33064: BlockingRecv error: recv got EOF from 127.0.0.1:33064 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:10:59.044180 30732 catalog_manager.cc:1259] Loaded cluster ID: ae021018df064682944fb7162f6c5f30
I20260923 15:10:59.044301 30732 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: loading cluster ID for follower catalog manager: success
I20260923 15:10:59.045156 30732 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: acquiring CA information for follower catalog manager: success
I20260923 15:10:59.045480 30732 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5fcd2ad0b95c4279a978eb451a95d77b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:10:59.100236 30762 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: acquiring CA information for follower catalog manager: success
I20260923 15:10:59.100543 30762 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5adcbd47b3ce4963918c7b81dc45b4e6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:10:59.199961 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:59.197321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33074 (local address 127.7.250.125:44847)
0923 15:10:59.197418 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:59.197421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:59.197476 (+ 55us) server_negotiation.cc:407] Connection header received
0923 15:10:59.197547 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:59.197548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:59.197595 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:59.197663 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:59.197808 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.198406 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.199200 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.199387 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.199438 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:59.199664 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:59.199772 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:59.199828 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:59.199876 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 15:10:59.199893 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":54,"threads_started":1}
I20260923 15:10:59.204679 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:59.202074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54866 (local address 127.7.250.124:38373)
0923 15:10:59.202215 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:59.202216 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:59.202225 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:10:59.202357 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:59.202358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:59.202394 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:59.202464 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:59.202636 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.203183 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.203938 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.204176 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.204234 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:59.204365 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:59.204466 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:59.204529 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:59.204549 (+ 20us) server_negotiation.cc:299] Negotiation successful
0923 15:10:59.204566 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260923 15:10:59.210342 30846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:59.206579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.7.250.126:39251)
0923 15:10:59.206711 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:59.206713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:59.206767 (+ 54us) server_negotiation.cc:407] Connection header received
0923 15:10:59.206868 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:59.206869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:59.206919 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:59.207031 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:59.207270 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.208101 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.209254 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.209576 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.209673 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:59.209874 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:59.209996 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:59.210060 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:59.210179 (+ 119us) server_negotiation.cc:299] Negotiation successful
0923 15:10:59.210203 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260923 15:10:59.210577 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:59.206780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.7.250.124:38373)
0923 15:10:59.206814 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:59.206815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:59.207103 (+ 288us) server_negotiation.cc:407] Connection header received
0923 15:10:59.207127 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:59.207128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:59.207140 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:59.207190 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:59.207384 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.208098 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.209723 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.210026 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.210104 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:59.210309 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:59.210344 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:59.210388 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:59.210472 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 15:10:59.210501 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 15:10:59.210965 30833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:10:59.206564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.7.250.125:44847)
0923 15:10:59.206608 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:10:59.206610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:10:59.206858 (+ 248us) server_negotiation.cc:407] Connection header received
0923 15:10:59.207009 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:10:59.207010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:10:59.207020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:10:59.207069 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:10:59.208200 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.208689 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.209962 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:10:59.210291 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:10:59.210352 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:10:59.210635 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:10:59.210674 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:10:59.210704 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0923 15:10:59.210836 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 15:10:59.210886 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:10:59.212404 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30481
I20260923 15:10:59.217840 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30556
I20260923 15:10:59.222934 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30631
2026-09-23T15:10:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2854 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:10:59 dist-test-slave-bb3x krb5kdc[30850](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:10:59 dist-test-slave-bb3x krb5kdc[30850](info): set up 2 sockets
Sep 23 15:10:59 dist-test-slave-bb3x krb5kdc[30850](info): commencing operation
krb5kdc: starting...
W20260923 15:11:00.284111 8169 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/test-user.keytab.
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:11:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.250.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
Entry for principal kudu/127.7.250.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab.
I20260923 15:11:00.414659 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41877
--webserver_interface=127.7.250.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41877,127.7.250.125:44837,127.7.250.124:44687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:11:00.496614 30863 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:11:00.496874 30863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:11:00.496905 30863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:11:00.496950 30863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:11:00.498118 30863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:11:00.498186 30863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:11:00.498209 30863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:11:00.498231 30863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:11:00.498291 30863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:11:00.499861 30863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41877,127.7.250.125:44837,127.7.250.124:44687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.126.keytab
--principal=kudu/127.7.250.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.126:41877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.250.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:11:00.500150 30863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:11:00.500384 30863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:11:00.505573 30869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:11:00.505517 30868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:11:00.505678 30863 server_base.cc:1034] running on GCE node
W20260923 15:11:00.505584 30871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:11:00.506170 30863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:11:00.506553 30863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:11:00.507745 30863 hybrid_clock.cc:648] HybridClock initialized: now 1790176260507720 us; error 47 us; skew 500 ppm
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:11:00.510591 30863 init.cc:377] Logged in from keytab as kudu/127.7.250.126@KRBTEST.COM (short username kudu)
I20260923 15:11:00.511049 30863 webserver.cc:460] Webserver started at http://127.7.250.126:41883/ using document root <none> and password file <none>
I20260923 15:11:00.511304 30863 fs_manager.cc:359] Metadata directory not provided
I20260923 15:11:00.511374 30863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:11:00.511524 30863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:11:00.512669 30863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data/instance:
uuid: "3daaf05a0394474694365c4055446671"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "76abf68f736ac32bbddb797faaa5bcdb"
server_key_iv: "a35ebf6c83f362d7b1cc26e88582c6b3"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.513027 30863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal/instance:
uuid: "3daaf05a0394474694365c4055446671"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "76abf68f736ac32bbddb797faaa5bcdb"
server_key_iv: "a35ebf6c83f362d7b1cc26e88582c6b3"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.514457 30863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:11:00.515239 30878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:11:00.515419 30863 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:11:00.515511 30863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal
uuid: "3daaf05a0394474694365c4055446671"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "76abf68f736ac32bbddb797faaa5bcdb"
server_key_iv: "a35ebf6c83f362d7b1cc26e88582c6b3"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.515617 30863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:11:00.533201 30863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:11:00.535265 30863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:11:00.535432 30863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:11:00.539518 30863 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.126:41877
I20260923 15:11:00.539574 30930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.126:41877 every 8 connection(s)
I20260923 15:11:00.539994 30863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/data/info.pb
I20260923 15:11:00.541101 30931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:11:00.542910 30931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:11:00.550043 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30863
I20260923 15:11:00.550153 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-0/wal/instance
I20260923 15:11:00.550407 8169 external_mini_cluster.cc:1428] Setting key 5c81dca55940e90197f15355808f96f1
I20260923 15:11:00.552763 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.543641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41877 (local address 127.0.0.1:48918)
0923 15:11:00.543810 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.543848 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.544342 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.544610 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.544613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.544858 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.545015 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.545018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.546087 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.546088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.546857 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.546859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.547335 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.547336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.547389 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.547887 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.547895 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.549261 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.551155 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.551163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.551165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.551440 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.551669 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.551670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.551671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.551790 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.552305 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.552312 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.552369 (+ 57us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.552447 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.552596 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":64,"threads_started":1}
I20260923 15:11:00.552836 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.543801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48918 (local address 127.7.250.126:41877)
0923 15:11:00.543928 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.543932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.543960 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:11:00.544476 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.544478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.544495 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.544580 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.545150 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.545978 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.546931 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.547242 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.547322 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.549339 (+ 2017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.549346 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.549346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.549352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.551041 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.551520 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.551523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.551523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.551601 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.551864 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.551865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.551865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.552227 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.552319 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.552510 (+ 191us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.552682 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":91,"thread_start_us":65,"threads_started":1}
I20260923 15:11:00.553496 30931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:11:00.553860 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.553725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46592)
0923 15:11:00.553776 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.553828 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:11:00.554006 30882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
I20260923 15:11:00.554487 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.554369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46594)
0923 15:11:00.554421 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.554455 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 15:11:00.554626 30931 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:44837: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.7.250.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab.
Entry for principal kudu/127.7.250.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab.
I20260923 15:11:00.578162 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:44837
--webserver_interface=127.7.250.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab
--principal=kudu/127.7.250.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41877,127.7.250.125:44837,127.7.250.124:44687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:11:00.603873 30931 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } attempt: 1
I20260923 15:11:00.604398 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.604232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46602)
0923 15:11:00.604304 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.604357 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:11:00.604842 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.604744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46604)
0923 15:11:00.604789 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.604821 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:11:00.604979 30931 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:44837: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
W20260923 15:11:00.660379 30938 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:11:00.660615 30938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:11:00.660633 30938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:11:00.660655 30938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:11:00.661999 30938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:11:00.662053 30938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:11:00.662067 30938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:11:00.662081 30938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:11:00.662092 30938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:11:00.663735 30938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41877,127.7.250.125:44837,127.7.250.124:44687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.125.keytab
--principal=kudu/127.7.250.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.125:44837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.250.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:11:00.664023 30938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:11:00.664268 30938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:11:00.669533 30944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:11:00.669577 30943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:11:00.669584 30946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:11:00.669840 30938 server_base.cc:1034] running on GCE node
I20260923 15:11:00.670090 30938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:11:00.670470 30938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:11:00.671633 30938 hybrid_clock.cc:648] HybridClock initialized: now 1790176260671624 us; error 46 us; skew 500 ppm
I20260923 15:11:00.673194 30931 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } attempt: 2
I20260923 15:11:00.673661 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.673497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46614)
0923 15:11:00.673590 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.673631 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260923 15:11:00.674131 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.673992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46628)
0923 15:11:00.674063 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.674101 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260923 15:11:00.674296 30931 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.125:44837: Network error: Client connection negotiation failed: client connection to 127.7.250.125:44837: connect: Connection refused (error 111)
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:11:00.674821 30938 init.cc:377] Logged in from keytab as kudu/127.7.250.125@KRBTEST.COM (short username kudu)
I20260923 15:11:00.675549 30938 webserver.cc:460] Webserver started at http://127.7.250.125:37617/ using document root <none> and password file <none>
I20260923 15:11:00.675853 30938 fs_manager.cc:359] Metadata directory not provided
I20260923 15:11:00.675931 30938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:11:00.676098 30938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:11:00.677910 30938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data/instance:
uuid: "f9146d1e3648435a965629345cc6ac4d"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "5428a348a0595fded144890433969934"
server_key_iv: "56ff4c3304269e4b184c5f613ff654f4"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.678480 30938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal/instance:
uuid: "f9146d1e3648435a965629345cc6ac4d"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "5428a348a0595fded144890433969934"
server_key_iv: "56ff4c3304269e4b184c5f613ff654f4"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.680527 30938 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260923 15:11:00.681742 30953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:11:00.681957 30938 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:11:00.682175 30938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal
uuid: "f9146d1e3648435a965629345cc6ac4d"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "5428a348a0595fded144890433969934"
server_key_iv: "56ff4c3304269e4b184c5f613ff654f4"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.682307 30938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:11:00.726770 30938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:11:00.729571 30938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:11:00.729769 30938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:11:00.734236 30938 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.125:44837
I20260923 15:11:00.734292 31005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.125:44837 every 8 connection(s)
I20260923 15:11:00.734714 30938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/data/info.pb
I20260923 15:11:00.735769 31006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:11:00.737445 31006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:11:00.744908 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 30938
I20260923 15:11:00.745054 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-1/wal/instance
I20260923 15:11:00.745299 8169 external_mini_cluster.cc:1428] Setting key 7e0289628a7375f4fb6ea32e19bcb31e
I20260923 15:11:00.749558 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.738314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48928 (local address 127.7.250.126:41877)
0923 15:11:00.738413 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.738416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.738632 (+ 216us) server_negotiation.cc:407] Connection header received
0923 15:11:00.739287 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.739288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.739306 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.739363 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.740069 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.740625 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.741748 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.741993 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.742084 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.745041 (+ 2957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.745042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.745043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.745053 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.747348 (+ 2295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.748010 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.748011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.748012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.748099 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.748526 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.748528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.748528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.748701 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.748753 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.749285 (+ 532us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.749428 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:11:00.749671 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.738247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41877 (local address 127.0.0.1:48928)
0923 15:11:00.738481 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.738538 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.739157 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.739440 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.739444 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.739729 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.739940 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.739944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.740738 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.740739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.741626 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.741628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.742120 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.742123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.742338 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.742970 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.742983 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.744893 (+ 1910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.747471 (+ 2578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.747482 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.747484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.747897 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.748249 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.748250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.748251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.748429 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.748812 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.748821 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.749035 (+ 214us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.749164 (+ 129us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.749383 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":119,"threads_started":1}
I20260923 15:11:00.750660 31006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.250.124@KRBTEST.COM; defaulting to no policy
I20260923 15:11:00.760598 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.751026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46636)
0923 15:11:00.751270 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.751284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.751337 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.751518 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.751519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.751705 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.751783 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.751785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.752844 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.752845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.753854 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.753858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.754570 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.754570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.754848 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.755306 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.755312 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.757006 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.759381 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.759382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.759382 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.759546 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.759770 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.759771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.759771 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.759825 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.760172 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.760174 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.760236 (+ 62us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.760329 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.760456 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226}
I20260923 15:11:00.760979 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.751109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46636 (local address 127.7.250.125:44837)
0923 15:11:00.751309 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.751314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.751345 (+ 31us) server_negotiation.cc:407] Connection header received
0923 15:11:00.751387 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.751390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.751404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.751501 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.751947 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.752780 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.753949 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.754470 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.754573 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.757100 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.757108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.757108 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.757138 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.759260 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.759643 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.759646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.759646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.759704 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.759896 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.759897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.759897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.760086 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.760186 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.760764 (+ 578us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.760858 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260923 15:11:00.761682 31006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:11:00.762156 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.761980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:56972)
0923 15:11:00.762042 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.762109 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.250.124@KRBTEST.COM" created.
W20260923 15:11:00.762382 30955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
I20260923 15:11:00.762856 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.762763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:56988)
0923 15:11:00.762802 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.762826 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 15:11:00.764367 31006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44687: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
I20260923 15:11:00.765439 30931 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } attempt: 3
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:11:00.773829 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.765836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46646)
0923 15:11:00.765902 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.765915 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.765969 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.766209 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.766210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.766471 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.766575 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.766578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.767574 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.767576 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.768544 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.768549 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.768978 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.768979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.769063 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.769545 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.769558 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.771011 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.772738 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.772739 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.772740 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.772921 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.773132 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.773133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.773133 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.773195 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.773469 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.773471 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.773522 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.773608 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.773724 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 15:11:00.773828 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.765879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46646 (local address 127.7.250.125:44837)
0923 15:11:00.766055 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.766057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.766066 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:11:00.766095 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.766096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.766110 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.766661 (+ 551us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.766705 (+ 44us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.767476 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.768643 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.768870 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.768975 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.771124 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.771125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.771126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.771134 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.772627 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.772996 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.772998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.772998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.773062 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.773267 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.773268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.773269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.773392 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.773466 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.773651 (+ 185us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.773732 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152}
I20260923 15:11:00.775314 30931 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:11:00.776326 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.776190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57002)
0923 15:11:00.776242 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.776268 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:11:00.776654 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.776581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57008)
0923 15:11:00.776609 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.776633 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260923 15:11:00.776786 30931 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44687: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.250.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab.
Entry for principal kudu/127.7.250.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab.
I20260923 15:11:00.785279 8169 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskOOn7ma/build/release/bin/kudu
/tmp/dist-test-taskOOn7ma/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:44687
--webserver_interface=127.7.250.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab
--principal=kudu/127.7.250.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.250.84:43295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.250.126:41877,127.7.250.125:44837,127.7.250.124:44687
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:11:00.807942 30931 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } attempt: 1
I20260923 15:11:00.808472 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.808333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57018)
0923 15:11:00.808400 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.808440 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:11:00.808832 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.808739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57024)
0923 15:11:00.808783 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.808807 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:11:00.808919 30931 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44687: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
I20260923 15:11:00.813589 31006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } attempt: 1
I20260923 15:11:00.814059 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.813875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57032)
0923 15:11:00.813935 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.814029 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 15:11:00.814440 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.814342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57040)
0923 15:11:00.814393 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.814418 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 15:11:00.814535 31006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44687: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
W20260923 15:11:00.877856 31013 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:11:00.878104 31013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:11:00.878130 31013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:11:00.878153 31013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:11:00.879503 31013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:11:00.879551 31013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:11:00.879567 31013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:11:00.879581 31013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:11:00.879628 31013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:11:00.881314 31013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.250.84:43295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.250.126:41877,127.7.250.125:44837,127.7.250.124:44687
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/krb5kdc/kudu_127.7.250.124.keytab
--principal=kudu/127.7.250.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.250.124:44687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.250.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:11:00.881595 31013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:11:00.881872 31013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:11:00.882673 31006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } attempt: 2
I20260923 15:11:00.883168 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.883017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57048)
0923 15:11:00.883084 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.883125 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:11:00.883585 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.883487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57062)
0923 15:11:00.883540 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.883567 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 15:11:00.883744 31006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44687: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
I20260923 15:11:00.884078 30931 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } attempt: 2
I20260923 15:11:00.884438 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.884326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57070)
0923 15:11:00.884380 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.884413 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:11:00.884799 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.884714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57072)
0923 15:11:00.884759 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.884784 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:11:00.884907 30931 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.250.124:44687: Network error: Client connection negotiation failed: client connection to 127.7.250.124:44687: connect: Connection refused (error 111)
W20260923 15:11:00.887914 31019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:11:00.887917 31021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:11:00.887892 31018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:11:00.888381 31013 server_base.cc:1034] running on GCE node
I20260923 15:11:00.888691 31013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:11:00.889011 31013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:11:00.890220 31013 hybrid_clock.cc:648] HybridClock initialized: now 1790176260890208 us; error 34 us; skew 500 ppm
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:11:00.893242 31013 init.cc:377] Logged in from keytab as kudu/127.7.250.124@KRBTEST.COM (short username kudu)
I20260923 15:11:00.893790 31013 webserver.cc:460] Webserver started at http://127.7.250.124:36135/ using document root <none> and password file <none>
I20260923 15:11:00.894044 31013 fs_manager.cc:359] Metadata directory not provided
I20260923 15:11:00.894091 31013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:11:00.894240 31013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:11:00.895386 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data/instance:
uuid: "89b8b27e4d9d41b68b694a69557c942d"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "11c6e6296d13a6b603a50a3a85099fb5"
server_key_iv: "f179ae69d0db66143e948c16e96d2f08"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.895766 31013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal/instance:
uuid: "89b8b27e4d9d41b68b694a69557c942d"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "11c6e6296d13a6b603a50a3a85099fb5"
server_key_iv: "f179ae69d0db66143e948c16e96d2f08"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.897193 31013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:11:00.898168 31028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:11:00.898391 31013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:11:00.898506 31013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data,/tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal
uuid: "89b8b27e4d9d41b68b694a69557c942d"
format_stamp: "Formatted at 2026-09-23 15:11:00 on dist-test-slave-bb3x"
server_key: "11c6e6296d13a6b603a50a3a85099fb5"
server_key_iv: "f179ae69d0db66143e948c16e96d2f08"
server_key_version: "encryptionkey@0"
I20260923 15:11:00.898567 31013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:11:00.916949 31013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:11:00.919461 31013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:11:00.919580 31013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:11:00.923552 31013 rpc_server.cc:307] RPC server started. Bound to: 127.7.250.124:44687
I20260923 15:11:00.923640 31080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.250.124:44687 every 8 connection(s)
I20260923 15:11:00.923983 31013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/data/info.pb
I20260923 15:11:00.925014 31081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:11:00.926646 31081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:11:00.931803 8169 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskOOn7ma/build/release/bin/kudu as pid 31013
I20260923 15:11:00.931938 8169 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskOOn7ma/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175983307587-8169-0/minicluster-data/master-2/wal/instance
I20260923 15:11:00.932225 8169 external_mini_cluster.cc:1428] Setting key 3beccc0347398c9c298f2010af23b59f
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:11:00.937247 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.927394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.7.250.126:41877)
0923 15:11:00.927462 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.927464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.927644 (+ 180us) server_negotiation.cc:407] Connection header received
0923 15:11:00.928099 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.928101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.928112 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.928184 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.928828 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.929741 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.931155 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.931505 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.931597 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.934167 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.934168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.934169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.934177 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.935730 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.936140 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.936142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.936142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.936209 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.936504 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.936504 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.936505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.936619 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.936674 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.936967 (+ 293us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.937156 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:11:00.937332 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.927323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.126:41877 (local address 127.0.0.1:48940)
0923 15:11:00.927485 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.927529 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.927992 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.928256 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.928259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.928503 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.928703 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.928707 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.929861 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.929863 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.931022 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.931026 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.931694 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.931696 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.931855 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.932568 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.932578 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.934077 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.935807 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.935814 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.935816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.936038 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.936285 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.936286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.936286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.936427 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.936682 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.936688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.936806 (+ 118us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.936929 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.937087 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:11:00.938472 31081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:11:00.942188 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.932814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57086 (local address 127.7.250.124:44687)
0923 15:11:00.932976 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.932981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.933000 (+ 19us) server_negotiation.cc:407] Connection header received
0923 15:11:00.933023 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.933029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.933047 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.933108 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.933766 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.934697 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.935586 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.936026 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.936110 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.938664 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.938671 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.938671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.938678 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.940755 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.941119 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.941123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.941123 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.941194 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.941428 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.941429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.941430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.941676 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.941826 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.941968 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.942068 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:11:00.946364 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.938733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.125:44837 (local address 127.0.0.1:46652)
0923 15:11:00.938782 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.938792 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.938827 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.939002 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.939003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.939255 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.939322 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.939325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.940176 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.940177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.941015 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.941018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.941548 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.941549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.941624 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.942040 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.942045 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.943528 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.945111 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.945112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.945113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.945281 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.945495 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.945496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.945496 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.945564 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.946009 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.946010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.946055 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.946146 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.946242 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 15:11:00.946362 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.938824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46652 (local address 127.7.250.125:44837)
0923 15:11:00.938895 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.938898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.938906 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:11:00.938932 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.938933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.938944 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.938998 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.939427 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.940060 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.941141 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.941471 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.941521 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.943626 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.943628 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.943628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.943635 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.945006 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.945359 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.945360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.945361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.945416 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.945819 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.945820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.945820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.945931 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.946017 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.946172 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.946243 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:11:00.946674 31081 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.126@KRBTEST.COM
I20260923 15:11:00.950695 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.943154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48954 (local address 127.7.250.126:41877)
0923 15:11:00.943218 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.943219 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.943226 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:11:00.943247 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.943248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.943257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.943328 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.943984 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.944449 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.945127 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.945333 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.945416 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.947859 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.947861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.947861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.947867 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.949450 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.949819 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.949820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.949820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.949883 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.950119 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.950120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.950121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.950239 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.950479 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.950517 (+ 38us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.950593 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.124@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:11:00.954195 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.947004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57102 (local address 127.7.250.124:44687)
0923 15:11:00.947046 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.947048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.947457 (+ 409us) server_negotiation.cc:407] Connection header received
0923 15:11:00.947517 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.947518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.947530 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.947598 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.947800 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.948313 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.949246 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.949453 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.949543 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.951444 (+ 1901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.951446 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.951446 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.951453 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.952916 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.953388 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.953389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.953389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.953450 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.953642 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.953643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.953643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.953767 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.953831 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.953963 (+ 132us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.954073 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":25}
I20260923 15:11:00.954295 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.946956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57102)
0923 15:11:00.947391 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.947404 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.947454 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.947600 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.947600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.947703 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.947733 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.947734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.948372 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.948373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.949160 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.949163 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.949561 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.949562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.949603 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.949971 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.949976 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.951380 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.953000 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.953001 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.953001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.953321 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.953523 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.953523 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.953523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.953580 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.953831 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.953832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.953870 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.953927 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.954072 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398}
I20260923 15:11:00.956105 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: Bootstrap starting.
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.250.125@KRBTEST.COM
I20260923 15:11:00.957237 31081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: Neither blocks nor log segments found. Creating new log.
I20260923 15:11:00.957631 31081 log.cc:824] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:11:00.958536 31081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: No bootstrap required, opened a new log
I20260923 15:11:00.959968 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.951700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.7.250.125:44837)
0923 15:11:00.951769 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.951771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.951780 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:11:00.951803 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.951804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.951815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.951873 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.952547 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.953132 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.954063 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.954490 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.954582 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.957083 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.957085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.957085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.957091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.958749 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.959127 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.959129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.959129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.959195 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.959423 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.959423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.959424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.959555 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.959731 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.959783 (+ 52us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.959859 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:11:00.960024 31081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:00.960179 31081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b8b27e4d9d41b68b694a69557c942d, State: Initialized, Role: FOLLOWER
I20260923 15:11:00.960258 31081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:00.960574 31081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:11:00.960598 31087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:00.960721 31087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:00.961835 8169 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260923 15:11:00.962462 31098 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:11:00.962525 31098 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:11:00.962852 30905 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3daaf05a0394474694365c4055446671"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48954
I20260923 15:11:00.962694 31013 master_runner.cc:186] Error getting master registration for 127.7.250.126:41877: OK, instance_id { permanent_uuid: "3daaf05a0394474694365c4055446671" instance_seqno: 1790176260538215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:11:00.964870 31013 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:11:00.975059 31006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } attempt: 3
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.125@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:11:00.983991 30931 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } attempt: 3
I20260923 15:11:00.984088 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.975505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57118)
0923 15:11:00.975563 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.975576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.975623 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.975850 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.975851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.976048 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.976118 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.976120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.977247 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.977248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.978024 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.978026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.978592 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.978593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.978653 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.979198 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.979204 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.980855 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.982879 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.982880 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.982881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.983118 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.983344 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.983346 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.983346 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.983415 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.983763 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.983764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.983813 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.983881 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.983985 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:11:00.984129 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.975557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.7.250.124:44687)
0923 15:11:00.975643 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.975646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.975685 (+ 39us) server_negotiation.cc:407] Connection header received
0923 15:11:00.975718 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.975719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.975733 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.975821 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.976204 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.977149 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.978116 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.978511 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.978591 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.980950 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.980952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.980952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.980961 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.982787 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.983201 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.983202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.983203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.983269 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.983482 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.983483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.983484 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.983692 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.983769 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.983928 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.984026 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:11:00.986219 31006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: Bootstrap starting.
I20260923 15:11:00.987066 31006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: Neither blocks nor log segments found. Creating new log.
I20260923 15:11:00.987403 31006 log.cc:824] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:11:00.988372 31006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: No bootstrap required, opened a new log
Sep 23 15:11:00 dist-test-slave-bb3x krb5kdc[30850](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790176260, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.250.126@KRBTEST.COM for kudu/127.7.250.124@KRBTEST.COM
I20260923 15:11:00.990705 31006 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:00.990865 31006 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9146d1e3648435a965629345cc6ac4d, State: Initialized, Role: FOLLOWER
I20260923 15:11:00.990952 31006 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:00.992031 31006 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:11:00.992010 31102 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:00.992152 31102 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:00.993144 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.984478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57128 (local address 127.7.250.124:44687)
0923 15:11:00.984590 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:00.984592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:00.984598 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:11:00.984617 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:00.984618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:00.984627 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:00.984685 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:00.986326 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.987046 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.987759 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.988028 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.988115 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.990122 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:00.990123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:00.990124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:00.990131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:00.991834 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.992284 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.992285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.992285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.992346 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:00.992558 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:00.992559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:00.992559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:00.992699 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:00.992765 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:00.992976 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 15:11:00.993047 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":127,"threads_started":1}
I20260923 15:11:00.993152 30933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:00.984284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.250.124:44687 (local address 127.0.0.1:57128)
0923 15:11:00.984349 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:11:00.984361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:11:00.984400 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:11:00.984729 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:11:00.984731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:11:00.984910 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:11:00.984956 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.984958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.987135 (+ 2177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.987136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.987662 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:11:00.987664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:11:00.988115 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:11:00.988116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:11:00.988180 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:00.988670 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:11:00.988676 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:11:00.990029 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:11:00.991922 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.991924 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.991924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.992161 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.992421 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:11:00.992422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:11:00.992423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:11:00.992484 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:11:00.992797 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:11:00.992798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:11:00.992845 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 15:11:00.992938 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 15:11:00.993047 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 15:11:00.994899 30931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671: Bootstrap starting.
I20260923 15:11:00.995651 30931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671: Neither blocks nor log segments found. Creating new log.
I20260923 15:11:00.995977 30931 log.cc:824] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671: Log is configured to *not* fsync() on all Append() calls
I20260923 15:11:00.994872 30938 master_runner.cc:186] Error getting master registration for 127.7.250.126:41877: OK, instance_id { permanent_uuid: "3daaf05a0394474694365c4055446671" instance_seqno: 1790176260538215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:11:00.998093 30938 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 15:11:00.994192 31113 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:11:00.998179 31113 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:11:00.999074 30931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671: No bootstrap required, opened a new log
I20260923 15:11:01.000545 30931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:01.000707 30931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3daaf05a0394474694365c4055446671, State: Initialized, Role: FOLLOWER
I20260923 15:11:01.000841 30931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:01.001236 30931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:11:01.001295 30905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 15:11:01.001360 30905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:11:01.002319 30905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:01.002585 30905 leader_election.cc:288] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [CANDIDATE]: Term 1 election: Requested vote from peers f9146d1e3648435a965629345cc6ac4d (127.7.250.125:44837), 89b8b27e4d9d41b68b694a69557c942d (127.7.250.124:44687)
I20260923 15:11:01.003302 30980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3daaf05a0394474694365c4055446671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f9146d1e3648435a965629345cc6ac4d"
I20260923 15:11:01.003297 31055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3daaf05a0394474694365c4055446671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "89b8b27e4d9d41b68b694a69557c942d"
I20260923 15:11:01.003435 30980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:11:01.003443 31055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:11:01.004766 31055 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daaf05a0394474694365c4055446671 in term 1.
I20260923 15:11:01.004779 30980 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3daaf05a0394474694365c4055446671 in term 1.
I20260923 15:11:01.005103 30882 leader_election.cc:302] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3daaf05a0394474694365c4055446671, f9146d1e3648435a965629345cc6ac4d; no voters:
W20260923 15:11:01.005862 31127 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:11:01.005921 31127 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:11:01.001181 31116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.006378 31116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:01.006489 31116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:11:01.006603 31116 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [term 1 LEADER]: Becoming Leader. State: Replica: 3daaf05a0394474694365c4055446671, State: Running, Role: LEADER
I20260923 15:11:01.006748 31116 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } }
I20260923 15:11:01.007602 31116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3daaf05a0394474694365c4055446671. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.007743 31116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: This master's current role is: LEADER
I20260923 15:11:01.008106 31135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:11:01.008236 30863 master_runner.cc:263] No leader master found from master 3daaf05a0394474694365c4055446671
I20260923 15:11:01.008291 30863 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:11:01.008419 31135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:11:01.010553 31055 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [term 1 FOLLOWER]: Refusing update from remote peer 3daaf05a0394474694365c4055446671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:11:01.010942 31116 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:11:01.012347 31087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3daaf05a0394474694365c4055446671. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.012441 31087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:01.012907 30980 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [term 1 FOLLOWER]: Refusing update from remote peer 3daaf05a0394474694365c4055446671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:11:01.013231 31116 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:11:01.014572 31102 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3daaf05a0394474694365c4055446671. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.014672 31102 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:01.015836 31116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.015946 31116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: This master's current role is: LEADER
I20260923 15:11:01.016517 31087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.016609 31087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:01.016949 31136 mvcc.cc:205] Tried to move back new op lower bound from 7332561965096488960 to 7332561965086007296. Current Snapshot: MvccSnapshot[applied={T|T < 7332561965096488960}]
I20260923 15:11:01.017676 31137 mvcc.cc:205] Tried to move back new op lower bound from 7332561965096488960 to 7332561965086007296. Current Snapshot: MvccSnapshot[applied={T|T < 7332561965096488960}]
I20260923 15:11:01.018757 31132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.018857 31132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671 [sys.catalog]: This master's current role is: LEADER
I20260923 15:11:01.019377 31102 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3daaf05a0394474694365c4055446671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3daaf05a0394474694365c4055446671" member_type: VOTER last_known_addr { host: "127.7.250.126" port: 41877 } } peers { permanent_uuid: "f9146d1e3648435a965629345cc6ac4d" member_type: VOTER last_known_addr { host: "127.7.250.125" port: 44837 } } peers { permanent_uuid: "89b8b27e4d9d41b68b694a69557c942d" member_type: VOTER last_known_addr { host: "127.7.250.124" port: 44687 } } }
I20260923 15:11:01.019515 31102 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:11:01.019680 31135 catalog_manager.cc:1347] Generated new cluster ID: f5a73ed73bc341a7ad6b94d0f37fdfb3
I20260923 15:11:01.019729 31135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:11:01.019874 31139 mvcc.cc:205] Tried to move back new op lower bound from 7332561965096488960 to 7332561965086007296. Current Snapshot: MvccSnapshot[applied={T|T < 7332561965096488960}]
I20260923 15:11:01.025874 30934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:01.014529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48968 (local address 127.7.250.126:41877)
0923 15:11:01.014600 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.014602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.015655 (+ 1053us) server_negotiation.cc:407] Connection header received
0923 15:11:01.015769 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.015770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.015783 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.015819 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.018574 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:01.019382 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:01.020661 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:01.020964 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:01.021042 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:01.022752 (+ 1710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:01.022753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:01.022754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:01.022761 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:01.024490 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:01.024911 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:01.024912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:01.024912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:01.024971 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:01.025205 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:01.025206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:01.025206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:01.025345 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:01.025447 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:01.025676 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 15:11:01.025769 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:11:01.034703 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:01.014926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.7.250.125:44837)
0923 15:11:01.014968 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.014970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.016797 (+ 1827us) server_negotiation.cc:407] Connection header received
0923 15:11:01.016907 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.016909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.016922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.016957 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.020695 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:01.021427 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:01.022676 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:01.023003 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:01.023054 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:01.031366 (+ 8312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:01.031368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:01.031368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:01.031378 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:01.033224 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:01.033674 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:01.033675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:01.033675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:01.033739 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:01.033952 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:01.033953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:01.033953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:01.034115 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:01.034218 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:01.034490 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 15:11:01.034589 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:11:01.043393 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:01.015301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57132 (local address 127.7.250.124:44687)
0923 15:11:01.015342 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.015344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.018693 (+ 3349us) server_negotiation.cc:407] Connection header received
0923 15:11:01.018790 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.018791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.018802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.018831 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.031517 (+ 12686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:01.032834 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:01.038407 (+ 5573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:01.038725 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:01.038801 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:01.040387 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:11:01.040389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:11:01.040389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:11:01.040397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:11:01.042094 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:01.042506 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:01.042507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:01.042508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:01.042564 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:11:01.042758 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:11:01.042759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:11:01.042759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:11:01.042890 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:11:01.042985 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:01.043205 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 15:11:01.043289 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:11:01.064291 31135 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:11:01.065069 31135 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:11:01.085884 31135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3daaf05a0394474694365c4055446671: Generated new TSK 0
I20260923 15:11:01.086118 31135 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:11:01.123559 31085 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.121792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.7.250.124:44687)
0923 15:11:01.121836 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.121839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.122848 (+ 1009us) server_negotiation.cc:407] Connection header received
0923 15:11:01.122958 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.122959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.122972 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.123004 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.123480 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57146: BlockingRecv error: recv got EOF from 127.0.0.1:57146 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:11:01.124392 30934 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.122043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48970 (local address 127.7.250.126:41877)
0923 15:11:01.122095 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.122097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.123832 (+ 1735us) server_negotiation.cc:407] Connection header received
0923 15:11:01.123937 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.123938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.123949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.123980 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.124327 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48970: BlockingRecv error: recv got EOF from 127.0.0.1:48970 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:11:01.125137 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.122546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.7.250.125:44837)
0923 15:11:01.122586 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.122588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.124615 (+ 2027us) server_negotiation.cc:407] Connection header received
0923 15:11:01.124713 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.124714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.124724 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.124756 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.125060 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46672: BlockingRecv error: recv got EOF from 127.0.0.1:46672 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:11:01.127568 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.126743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.7.250.125:44837)
0923 15:11:01.126777 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.126779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.126974 (+ 195us) server_negotiation.cc:407] Connection header received
0923 15:11:01.127139 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.127141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.127164 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.127200 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.127522 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46686: BlockingRecv error: recv got EOF from 127.0.0.1:46686 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260923 15:11:01.131125 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.130133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46694 (local address 127.7.250.125:44837)
0923 15:11:01.130174 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.130176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.130506 (+ 330us) server_negotiation.cc:407] Connection header received
0923 15:11:01.130620 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.130621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.130633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.130683 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.131072 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46694: BlockingRecv error: recv got EOF from 127.0.0.1:46694 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:11:01.135392 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.134501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.7.250.125:44837)
0923 15:11:01.134542 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.134544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.134795 (+ 251us) server_negotiation.cc:407] Connection header received
0923 15:11:01.134905 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.134906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.134918 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.134950 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.135337 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46700: BlockingRecv error: recv got EOF from 127.0.0.1:46700 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 15:11:01.141712 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.140776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.7.250.125:44837)
0923 15:11:01.140818 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.140820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.141090 (+ 270us) server_negotiation.cc:407] Connection header received
0923 15:11:01.141201 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.141202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.141215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.141250 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.141649 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46712: BlockingRecv error: recv got EOF from 127.0.0.1:46712 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:11:01.152406 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.151293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46720 (local address 127.7.250.125:44837)
0923 15:11:01.151343 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.151345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.151666 (+ 321us) server_negotiation.cc:407] Connection header received
0923 15:11:01.151785 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.151787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.151800 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.151833 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.152339 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46720: BlockingRecv error: recv got EOF from 127.0.0.1:46720 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:11:01.170907 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.169836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.7.250.125:44837)
0923 15:11:01.169884 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.169886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.170215 (+ 329us) server_negotiation.cc:407] Connection header received
0923 15:11:01.170378 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.170380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.170393 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.170427 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.170838 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46732: BlockingRecv error: recv got EOF from 127.0.0.1:46732 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 15:11:01.205226 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.204399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46742 (local address 127.7.250.125:44837)
0923 15:11:01.204455 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.204457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.204645 (+ 188us) server_negotiation.cc:407] Connection header received
0923 15:11:01.204732 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.204733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.204745 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.204793 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.205161 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46742: BlockingRecv error: recv got EOF from 127.0.0.1:46742 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:11:01.271440 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.270557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.7.250.125:44837)
0923 15:11:01.270663 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.270666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.270783 (+ 117us) server_negotiation.cc:407] Connection header received
0923 15:11:01.270880 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.270881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.270900 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.270969 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.271391 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46746: BlockingRecv error: recv got EOF from 127.0.0.1:46746 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260923 15:11:01.401876 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.400915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.7.250.125:44837)
0923 15:11:01.400975 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.400977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.401143 (+ 166us) server_negotiation.cc:407] Connection header received
0923 15:11:01.401240 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.401241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.401252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.401306 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.401790 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46756: BlockingRecv error: recv got EOF from 127.0.0.1:46756 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 15:11:01.659865 31010 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:11:01.659209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.7.250.125:44837)
0923 15:11:01.659293 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:01.659295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:01.659305 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:11:01.659335 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:01.659336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:01.659348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:01.659428 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:11:01.659775 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46770: BlockingRecv error: recv got EOF from 127.0.0.1:46770 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:11:01.963047 31098 catalog_manager.cc:1259] Loaded cluster ID: f5a73ed73bc341a7ad6b94d0f37fdfb3
I20260923 15:11:01.963161 31098 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: loading cluster ID for follower catalog manager: success
I20260923 15:11:01.963972 31098 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: acquiring CA information for follower catalog manager: success
I20260923 15:11:01.964325 31098 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 89b8b27e4d9d41b68b694a69557c942d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:11:01.998659 31113 catalog_manager.cc:1259] Loaded cluster ID: f5a73ed73bc341a7ad6b94d0f37fdfb3
I20260923 15:11:01.998773 31113 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: loading cluster ID for follower catalog manager: success
I20260923 15:11:01.999639 31113 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: acquiring CA information for follower catalog manager: success
I20260923 15:11:01.999979 31113 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f9146d1e3648435a965629345cc6ac4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:11:02.176370 31212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:02.173113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.7.250.125:44837)
0923 15:11:02.173287 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:02.173289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:02.173304 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:11:02.173396 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:02.173397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:02.173455 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:02.173545 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:11:02.173830 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.174451 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.175346 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.175557 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.175630 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:02.175803 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:11:02.175917 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:11:02.176034 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:02.176180 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 15:11:02.176206 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":64,"threads_started":1}
I20260923 15:11:02.181604 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:02.178567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57160 (local address 127.7.250.124:44687)
0923 15:11:02.178681 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:02.178684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:02.178694 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:11:02.178789 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:02.178789 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:02.178850 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:02.179083 (+ 233us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:11:02.179193 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.179738 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.180610 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.180861 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.180983 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:02.181179 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:11:02.181313 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:11:02.181403 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:02.181472 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 15:11:02.181497 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":56,"threads_started":1}
I20260923 15:11:02.186733 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:02.183553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57170 (local address 127.7.250.124:44687)
0923 15:11:02.183598 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:02.183600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:02.183790 (+ 190us) server_negotiation.cc:407] Connection header received
0923 15:11:02.183825 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:02.183826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:02.183840 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:02.183894 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:11:02.184065 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.184686 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.185819 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.186154 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.186232 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:02.186449 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:11:02.186495 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:11:02.186559 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:02.186610 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 15:11:02.186641 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:11:02.187501 31212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:02.183674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.7.250.125:44837)
0923 15:11:02.183729 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:02.183731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:02.183892 (+ 161us) server_negotiation.cc:407] Connection header received
0923 15:11:02.184009 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:02.184011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:02.184031 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:02.184097 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:11:02.184240 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.185027 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.186483 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.186892 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.186979 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:02.187135 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:11:02.187175 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:11:02.187256 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:02.187372 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 15:11:02.187402 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 15:11:02.187528 31224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:11:02.183465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.7.250.126:41877)
0923 15:11:02.183578 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0923 15:11:02.183580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:11:02.183774 (+ 194us) server_negotiation.cc:407] Connection header received
0923 15:11:02.183831 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:11:02.183832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:11:02.183901 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:11:02.184160 (+ 259us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:11:02.184398 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.185218 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.186221 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:11:02.186571 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:11:02.186649 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:11:02.187126 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:11:02.187247 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:11:02.187327 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:11:02.187384 (+ 57us) server_negotiation.cc:299] Negotiation successful
0923 15:11:02.187413 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":52,"threads_started":1}
I20260923 15:11:02.189735 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30863
I20260923 15:11:02.196202 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 30938
I20260923 15:11:02.201328 8169 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskOOn7ma/build/release/bin/kudu with pid 31013
2026-09-23T15:11:02Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2978 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5832 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278899 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST